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PREFACE 



When I was asked originally to prepare a monograph on indicators of -4ucational 
efficiency, my initial response was negative. The excellent work done on the topic of 
educauonai efficiency over the last decade by such economists as Mary Jean Bowman, Enc 
Hanushck. and Henry Levin has recendy been supplemented by two extensive World Bank 
survey papers by Bill Fuller and Marlaine Lockheed.* This raised a serious question both 
as to the need for and potential value-added of a monograph such as this one. The issue was 
reinforced by the fact that my own papers. Internal Efficiency and the African School 
(prepared for the World Bank) and— coauthored with David Chapman— the Evaluation of 
Efficiency in Educational Development Activities (prepared for the Improving the Effi- 
ciency of Educational Systems project), contained much of what I wished to say about the 
policy application of efficiency concepts to the evaluation of educational programs and 
projects. 

However, in response to the entreaties of my lEES colleagues (most notably. David 
Sprague and Joan Claf fey of the Agency for International Development and Robert Morgan 
of Florida State University) I agreed to accept the responsibility for preparing a paper on 
indicators of educational effectiveness and efficiency. The paper was to be designed as a 
companion piece to the Chapman AVindham evaluation monograph. There were four major 
justifications offered to me for preparing such a paper: 

L the opportunity to synthesize a large portion of the literature on educational 
efficiency in a form that would make it accessible and useful to a wide range of 
non-economist professionals; 

2. to distill from the varied experiences of the lEES project some of the lessons 
learned in terms of the opportunities and limitations encountered in promoting 
efficiency enhancement in a developing nation setting; 

3. to detail how the products of efficiency analysis can inform the development of 
educational management information systems and data-based argument among 
educational administrators and policymakers; and 



* Older researchers will remember that during previous incarnations of US AID interest m 
educational indicators. Sclma Mushkin and her colleagues Fassil Kiros and Bradley 
Billings produced several reports on this topic. The most noteworthy are Kiros. 
Mushkin. and Billings. Educational Outcome Measurement in Developing Countries 
(1975) and Mushkin. Educational Outcomes and Nutrition (1977). The present effort 
recognizes the seminal nature of this early work and attempts to assure that the 
contribuuons of Mushkin, cu al.. arc not lost to the current generation of analysts and 
policymakers. 
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4. to create a basic text from which a set of management training manuals could be 
developed to promote the understanding and appropriate use of efficiency 
concepts in educational decisionmaking. 

With these four justifications as goals, work on this monograph began in April 1987, 
Two immediate problems were encountered. The first was the need to compromise 
between the level of detail and analytical sophistication that exists in the economic 
literature and the types of data and issues faced by educational decisionmakers in mosr. 
developing nations. Because the large majority of decisionmakers in such settings are not 
formally trained in economics, there was a derived need to decide how much background 
in economic theory was required to present the efficiency concepts in a meaningful manner. 
Based on my personal experience in training and counterpart situations with colleagues in 
such settings, the material presented here has been designed to require no formal experience 
in economics but to rely heavily on the reader's intelligence, industriousness, and openness 
to new concepts. It was felt that some concepts might prove redundant to the experience 
or training of some decisionmakers; however, when these cases occur, it was felt that it is 
easier for these individuals to skip over a section than it would be for the other readers to 
understand the use of the concepts without some discussion the concept's assumptions and 
derivation. 

The second compromise required was between a personal desire to advocate the 
increased use of efficiency concepts in educational planning and evaluation activities and 
a professional responsibility to couch such advocacy in terms of the constraints that exist 
in the application of efficiency concepts to the real world of educational decisionmaking. 
However, it should be su-esscd that many who express concern about the emerging 
emphasis on efficiency applications to education are confusing efficiency with fiscal 
reductions. This monograph tries to resolve this confusion and to distinguish self-interested 
objections to educational accountability from legitimate concerns over any excessively 
mechanistic approach to the crucial decisionmaking concerning the lives of students, 
teachers, and parents. 

As has already been indicated by the comments of my colleagues who reviewed the first 
draft of the monograph, not everyone will agree with where I have drawn the line between 
the interests of economists and those of educational decisionmakers or the position I have 
taken between advocacy and the promotion of skepticism. My consolation lies in the fact 
that the reviewers disagreed as much among themselves as with me on these issues and that 
no solution existed to satisfy all of them. 

Since this paper draws on my experiences in a variety of nations over the last fifteen 
years, any list of acknowledgements must be incomplete. I will begin by repeating my debt 
to Drs. Sprague, Claffey and Morgan for their encouragement that I undertake this project: 
I hope they have a minimum of regrets now that it is done. I owe a special debt to those 
colleagi:es who reviewed the draft version of the monograph in detail (Stephen Hoenack 
of the University of Minnesota, Donald Winklw of the University of Southern California, 
S. Thiagarajan of the Institute for International Research, Frank Famer of the Worid Bank, 
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and Dan Levy. David Chapman, and Frances Kemmerer of the State University of New 
York at Albany). They, of course, are not responsible for any failures by me to incorpora*^ 
their comments or to do justice to their many excellent suggestions. 



In addition. I am very appreciative of the support provided by colleagues such as Gary 
Thciscn. Mark Rilling, and Frank Method of the Agency for International Development; 
Stephen Hcyneman. Marlaine Lockheed, and Jacques Hallak of the V/orld Bank; Bikas 
Sanyal of the International Institute for FxJucational Planning; Jack Bock. Jerry Messec. 
Peter Easton. and Steven Klces of Florida State University; Willie Howard of Howard 
University; Victor Cicutat and Mary Pigozzi of the Institute for International Research; 
Valerie Janesick of the University of Hawaii; and all of my colleagues at the State 
University of New York but with special appreciation to Philip Foster. Warren Ilchman, 
Robert Koff. Alan Purvis. Jerry Strudwick. and Frederick Dembowski. Also I wish to 
extend my strong appreciation to my international colleagues on ihelEES project advisory 
committee: Minister Othello Gongar of Liberia. Deputy Minister Abdul Garada of the 
Yemen Arab Republic. Mr. Jakes Swartland of Botswana. Pak Moegiadi of tlie Republic 
of Indonesia. Mr. Ali Gaal of the Somali Democratic Republic; and to Mr. Ma Weixiang 
of the Slate Education Commission and Mr. Cai Pei-Yi of the Shanxi Province Education 
Department. People's Republic of China. Whether purposefully or inadvertently, all of 
these individuals have profoundly affected the manner in which I approach the economic 
analysis of education. 

My colleague. Dr. Kcmmcrer, deserves special commendation in another respect as 
well; as Institutional Coordinator for the lEES project at the Sute University of New York 
at Albany she has organized the administrative office in such a manner (hat the logistics 
of this volume were handled expeditiously even though I was away from Albany during 
most of the period of the volume's preparation. The two most valuable persons involved 
in this whole production process were Ms. Natalie WaschuU. EEES Project Administrative 
Assistant, and Ms. Catherine Wightman . Project Secretary. Their work on this activity was 
exceptional in every way — which for them is an unexceptional occurrence. 

Finally. Imustacknowlcdgethedebtowed to Ms. Jeannette Windham. Her editing skills 
arc sorely tested in attempts to co^.lrol my abhorent spelling, con volute 1 sentence structure, 
and perverse preference for alliteration. However, more important than these efforts are her 
continued patience, support, and affection without which neither this paper nor any other 
meaningful task could be accomplished. 
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CHAPTER ONE 



THE CONCEPT OF EDUCATIONAL EFFICIENCY: AN INTRODUCTION 



The purpose of this monograph is to identify appropriate quantitative indicators of 
educational effectiveness and efficiency and to discuss how such indicators should be used 
in assessing education at multiple levels of the educational svstem. These statistical 
indicators of educational effectiveness and efficiency are requiied to document the present 
status of educational activities, to establish alternative goals for the education and human 
resources (EHR) system in terms of how it should appear at some future time, and to operate 
as benchmarks to define systemic progress toward better utilization of existing resources 
by the educational system or by individual educational organizations. 

The primary audiences for whom this monograph was written are the mid-level planners 
and evaluators in developing nations responsible for educational decisionmaking as well 
as the un iversity , government, and other advisors who assist these personnel. For some, the 
volume may serve as a self-instructional text; for others, it will prove more useful as a 
reference work. To increase the potential contribution of this volume, educational 
management training modules will be developed from this monograph. These modules will 
be designed for use in both group-instruction and self-instruction settings. 

The secondary audience for this volume is much wider and is inclusive, specifically, of 
education and evaluation professionals concerned with resource allocation in education 
and education's relationship to personal and national development and, generally, of any 
educationalists, social scientists, or other parties interested in the status of the application 
of economic concepts to educational analysis. Because of these multiple audiences, some 
sections of the volume may be in excessive detail for some readers already familiar with 
the material while in other sections, readers may wish to refer to the cited literature for a 
more detailed introduction to concepts with which they are unfamiliar. However, the 
volume is designed for use as a self-contained presentation of the issues of educational 
effeciivencss and efficiency since many readers may not have the time or access to other 
resources required to supplement this volume. It is hoped ihiat, while any individual reader 
may desire less of some discussions and/or more of others, the large majority of readers will 
find the volume useful and adaptable to their own training, experience, and professional 
needs. 

This monograph is designed as a companion volume to Chapman ar J Windham, The 
Evaluation of Efficiency in Educational Development Activities (1986). That monograph 
examined issues related to the design and conduct of program and project evaluation of 
activities that have the enhancement of educational efficiency as a goal. While it dealt 
extensively with the context, techniques, and processes of efficiency evaluation, the prior 
monograph did not deal in detail with the alternative means of operationalizing effective- 
ness or efficiency concepts. While this monograph is designed to be of benefit as an 
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independent volume, greater value will be derived by those familiar with the concepts and 
iss':cs treated in the Chapman and Windham monograph. 

In the last decade there has been a great increase in the attention paid to efficiency issues 
in regard to the role education can play in development (e.g., Windham, 1982B; Psacha- 
ropoulos and Woodhall, 1985; Windham and Wang, 1986). Thib increased attention has 
been brought about by the constrained fiscal conditions under which most developing 
nations are forced to operate and the heightened demand in these nations for resources from 
the EHR sector itself, from other social service sectors, and from the infrastructure sectors 
(e.g., transport and communications). Within this fiscal environment, the debate over 
efficiency issues has evolved into three forms of discussion: (1) rhetorical, (2) conceptual, 
and (3) practical. 

The rhetorical discussion of educational efficiency is best characterized by the treat- 
ments found in most national planning documents and the policy papers of the international 
donor agencies. Here, ^'efficiency" is rarely operationalized and even when used as a 
general concept, it is often unclear whether efficiency is meant to exist as a goal in and of 
itself or as a means to some other end. However, "efficiency" normally is assumed to be an 
inherently good thing and efficiency enhancement activities often are cited as a means of 
increasing the availability of funds required to improve educational access and/or quality. 

Within the rhetorical discussion of efficiency there is a minority view that is less 
supportive of the concept. The "efficiency movement" is viewed with suspicion by those 
who fear that educational efficiency will manifest itself primarily in the forms of lower 
fiscal allocations and reduced unit costs. Again, the "efficiency" standard is rarely defined 
by these critics who tend generally to oppose most encroachments by economists and 
financial analysts into the educational domains of pedagogues, administrators, and poli- 
cymakers. 

While much of the policy debate over -educational efficiency has been conducted at this 
first level of abstraction, economists in the last decade often have concentrated on equally 
abstract conceptual and definitional distinctions at the expense of pragmatic issues of 
relevance to administrators and policymakers. The economists' focus on taxonomic issues 
(internal versus external efiiciency, private versus social costs and effects, technological 
versus economic efficiency) has been useful to those noneconomists who have taken the 
time to master the terminology; unfortunately, few have been persuaded to do so because 
most educational administrators and policymakers operate in an environment that does not 
always allow for the fine distinctions and deliberations called for by the economic 
literature 

Thus, in the last decade the discussion of educ Jonal efficiency has been balanced 
between these polar forms of abstraction: the practitioners' undefineduseof efficiency as 
a totem-word and the economists* multiple use of efficiency as a context-specific concept. 
Inadequate attention has been paid by both groups to practical applications of the efficiency 
concept to educational activities. Practitioners often are uncomfortable in discussing these 
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issues because ihey feel they are at a disadvantage relative to economists in specifying 
operational measures of efficiency and in understanding the interpretive biases inherent to 
any such practical definitions. Similarly, economists have manifested a reluctance to 
abandon the purity of their conceptual definitions to deal with issues of practical specifi- 
cation and quantification. As will be explained later in this monograph, any operational 
definition of educational efficiency is subject to legitimate questioning. Any economists 
who advocate or appear to advocate particular efficiency measures are open to criticism 
from their colleagues for the conceptual inadequacy of a particular measure or the 
inadequacy of the form of its quantification. 

This monograph will focus on the practical aspects of introducing effectiveness and 
efficiency concepts and measures into the deliberations of educational practitioners — 
especially administrators, planners, and policymakers. While the discussion will originate 
from and be based upon the conceputual Definitions taken from economics, the proposed 
indicators and their uses will be judged primarily in terms of their appropriateness in the 
settings in which most educational decisions are made. Decisionmakers must recognize 
that they never have all of the information they need or all of the time they want: 
decisionmaking in educational efficiency requires a forced trade off of the quantity and 
quality of information versus the timeliness and effectiveness of decisions. 

The remainder of this monograph is arranged in four major parts. The immediately 
succeeding section will deal with the conceptual and definitional issues related to the 
measurement of educational effectiveness and efficiency. The appropriateness of the 
application of the efficiency metaphor to education will be reviewed and specific 
definitions for common terms will be proposed. The second major section of the 
monograph will deal with indicators of educational effectiveness; one chapter wiil deal 
with input and process measures and a second with output and outcome measures. In the 
discussion, the various indicators will include both those that are specifiable in financial 
terms and those that can be expressed in quantifiable but not financial terms. Also, the role 
for qualitative indicators in efficiency analysis will be discussed. The third major section 
of the monograph will present a review of basic cost issues and will demonstrate how 
efficiency analysis is conducted under four alternative forms: benefit-cost analysis, cost- 
cffcctivcncss analysis, least-cost analysis, and cost-utility analysis. 

The final major section of the monograph will attempt to assess the policy relevance of 
indicators of educational effectiveness and efficiency as they relate to the development and 
use of educational management information systems The discussion wUl focus upon the 
use of these indicators in policy decisions, the constraints and facilitators in the use of 
efficiency data, and the prospects for increased ani'or improved use of effectiveness or 
efficiency data in the planning and operation of educational and human resources 
development activities. The monograph will conclude with a brief review ar:d a set of 
recommendations of actions needed to improve the practical relevance of etiiciency 
considerations to educational systems and institutions. 
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Before proceeding to the main text of the monograph, it is necessary to clarify some of 
the assumptions and emphases that have structured this presentation. There are seven main 
statements that should help with this clarification: 

L There is an emphasis upon collective and public decisionmaking in regard to 
education but attention also will be given to the use of indicators in the support of 
individual decisionmaking; 

2. There is an emphasis on applications of educational efficiency indicators within 
developing natio.is (especially those in Africa and Asia) but the preponderance of 
the discussion has equal relevance to developed nations; 

3- The focus of the discussion and the preponderance of examples will be on primary 
and secondary education (including vocational/technical/ commercial alterna- 
tives) but, except where noted, the discussion also would apply to pre-primary, 
post-secondary, and both formal and nonformal adult education and training 
activities; 

4. Efficiency indicators will be dealt with within the context of the need to create and 
utilize comprehensive educational management information systems within 
developing nations; 

5- There will be a full discussion of the responsibility for efficiency indicators 
specifically and educational management information systems generally them- 
selves to be cost-effective and responsive to the fiscal conditions of the nation; 

6. The monograph's emphasis will be on the application of lesson's learned from the 
experiences of the Improving the Efficiency of Educational System's (lEES) 
project with a major secondary focus on the larger educational development 
literature; and 

7. All discussion will have as its goal the practical and operational aspects of 
efficiency assessment within the context of the efficiency concept's advantages 
and limitations. 
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CHAPTER TWO 



DEFINITIONAL AND METHODOLOGICAL ISSUES RELATED TO THE 
CONCEPT OF EDUCATIONAL EFHCIENCY 



The puqx)se of this chapter is to provide an introduction to the nature of the concept of 
efficiency, as currently used by most economists, and the advantages and disadvantages 
one encounters in applying llie concept to an activity as internally complex and contextu- 
ally diverse as education. In this chapter, a set of definitions will be established that will 
serve as the basic terminology used in the subsequent discussion of specific indicators of 
educational effectiveness and efficiency. The discussion will introduce the four major 
forms of efficiency analysis applied to education; benefit-cost, cost-effectiveness, cost- 
utility, and least-cost models will be presented. The chapter will conclude witii a review 
of five major limitations that exist in attempts to apply the economists' models of cost and 
productivity to education. 

It is surprising to most non-economists to learn that the concept of efficiency is, in fact, 
a relatively new emphasis within the lexicon of economics. Schumpeter's History of 
Economic Analysis (1966), the standard for tiie treatment of tiie development of Western 
economic thought, has not a single index reference to efficiency. Part of tiie reason for this 
earlier lack of overt attention was tiiat tiie efficiency concept was implicit to tiie market 
models developed by Western economists from tiie late 1 700s up to tiie 1930s. Only in the 
last fifty years has great attention been directed toward issues of measurement and 
empirical testing of the deductively derived theories of neoclassical economics (Johnson, 
1975).* The resuii of this new emphasis on quantification has been to raise the issues of 
the opcrationalization and measurement of the economic variables. The economist no 
longer can be satisfied simply to slate that under a given budget, efficiency exists, for a 
producer when the marginal cost of an output from a production process equals the output's 
marginal revenue product or for a consumer when the ratio of the marginal costs of all 
consumption items to their marginal utility are equal. Without debating tiie contribution 
that these abstract models (and tiie neoclassical insistence upon defining equilibria as 
optima) have had for understanding social and market phenomena, there has been a 
recognized need to produce a practical and adaptable form of efficiency tiiat can advance 
the management of private and social enterprise. 

The economic concept of efficiency is a metaphor borrowed from engineering relation- 
ships. In any technical process efficiency is defined as existing where tiie desired mix of 
outpu'^s (effectiveness) is maximized for a given level of inputs (cost) or where inputs 
are minimized for a desired mix of outputs. It is important to rccognizxi from tiiesc 
definitions that the concept of effectiveness (how well or to what extent tiie desired outputs 



* Schwartz and Bcrney (1977) offer an excellent set of discussions dealing witii the 
neoclassical economists' approach to the efficiency concept. 
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arc achieved) is subsumed in the concept of efficiency (effectiveness relative to cost). In 
the following sections of this monograph, the term effectiveness will be used when 
indicators represent outputs or output proxies (input or process variables and outcomes) and 
efficiency when the indicators represent a comparison of effectiveness with costs. In all 
cases efficiency is a more inclusive term and implies both effectiveness and cost consid- 
erations. 

If the definition of efficiency is specified in terms of physical quantities only, one has 
a definition of technological efficiency. If one modifies the concept to take into account 
utility* or monetary measures, a definition of economic efficiency is derived. Economic 
efficiehcy is defined as existing when the value of all outputs is maximized for a given 
cost of an inputs or where the cost of all inputs for a given value of all inputs. Both of 
the efficiency concepts, technological and economic, appear both rational and intuitively 
obvious. What is less obvious is how to measure inputs and outputs so one may know when 
efficiency exists and. in the case of economic efficiency, to know what values (costs or 
prices) to assign to inputs and outputs to avoid biasing the identification of efficiency. 

In a competitive market situation all firms must strive to achieve efficiency because the 
inability or unwillingness to do so will mean that their competitors can charge lower prices 
and drive the "inefficient" firms out of the market. Efficiency in a competitive market is 
therefore a sclf-monitoring and sclf-equilibriating process. Since firms in a competitive 
market are. by definition, small relative to the total market, the individual firms have no 
effect on the cost of inputs or the prices of their products. Thus, economic efficiency can 
be defined in a non-arbitrary manner. 

Unfortunately for those who prefer objective, mechanistic decision processes, the con- 
ditions of the competitive market are increasingly rare in general and simply do not exist 
in regard to the education and training systems of most nations. As will be developed here, 
the abandonment of the competitive assumptions docs not reduce the importance of the ef- 
ficiency concept; however, it does force those who wish to use it to deal with less than ideal 
proxies for their conceptual variables and to accept subjective responsibility for judgments 
concerning the values of inputs and outputs.** 

In almost all situations, education in developing nations is either a monopoly function 
of government or else government exists as a major financier and regulator of the 



* The economic concept of utility is dealt with at a later point. For the moment it is 
necessary to understand only that utility refers to perceived satisfaction or happiness. 

** Klees (1984) asserts that the requirement that market prices reflect efficiency is similar 
to the econometric condition that regression coefficients represent causal impact. He 
notes that "... both necessitate the fulfillment of relatively few, but totally unattainable, 
conditions and both have little practical guidance to offer on how inaccurate these 
indicators are when the necassary conditions do not hold../* 
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educational activity. In shifUng from the model of a competitive market to one of a 
bureaucratic management or regulatory system, one loses the self-monitoring and self- 
equilibriating characteristics that assured efficiency in the competitive situation. To 
replace them one must turn either to legal rules or bureaucratic incentives that are designed 
to achieve an approximation of efficiency. 

Some economists and many non-economists have questioned the propriety of transpos- 
ing the efficiency concept from a technical setting to a social or behavioral one (Klees. 
1984). A more appropriate question might have been whether it is possible and justified 
to transpose tile concept of competitive efficiency to a non-competitive context Itisclear. 
however, tiiat regardless of die philosophical uncertainty over the propriety of this 
transposition, the last ten years have seen a rapid escalation in attention paid to efficiency 
issues related to educational finance and management; and this increased attention has 
occurred in both socialist and market economies. 

If Uie result of Uiis increased attention to efficiency is tiiat more and or better educational 
benefits are obtained for a given level of expenditure then die use of the efficiency concept 
will be justified. If the result is hat educational planners and managers use economic 
models and jargon as a shield for Oieir biases and subjective judgments, then die use of die 
efficiency concept will not have served a legitimate purpose. It is important to understand 
diat die efficiency concept is a neutral device; it is die definition and valuation of its 
components (inputs, processes, outputs, and outcomes) diat will determine whedier die 
current attention focused on efficiency is a positive or negative contribution to educational 
development 

In proceeding to establish a basic glossary of efficiency terminology, it i:. useful to 
discuss die concepts of production and utility diat underly die practical discussion that 
follows. This discussion of theory is presented as a foundation for die later practical 
discussions. While it is possible for one to benefit from die subsequent practical discourse 
without an understanding of this theoretical foundation, one cannot claim to understand die 
efficiency issue fully without an appreciation of die concepts of production and utility. 

The production process for education, for which die major factors are depicted in 
simplified form in figure one. consists of four main parts: inputs, process, outputs, and 
outcomes. In figure one examples are given of die types of observable and measurable 
variables diat may be classified as belonging widiin each stage. Inputs are die resources 
used in die production activity: for educational production, inputs may be divided into die 
general categories of student characteristics, school characteristics, teacher characteristics, 
instructional material and equipment characteristics, and facilities characteristics. In each 
case die term "characteristics" refers to die availability of a resource, its nature and quality, 
and its manner and rate of utilization. 

For example, an important teacher characteristic would be die teacher's mastery of die 
subject matter (e.g., madiematics) for which die teacher is responsible. The effect of 



18 



7 



Chapter 2 



teacher subject matter competence on the education production process will depend on the 
existence of some measurable level of competence; its nature (the areas of mathematics 
skill mastered) and quality (the degree of competence); and its manner and rate of 
utilization (the means by which a unit of teacher time or effort is combined with other 
resources including student time and effort). 

The process stage of educational production refers to the means by which educational 
inputs are transformed into educational outputs. Often the term educational technology 
is used to refer to a specific process for promoting educational outputs; examples of 
educational technologies are classroom lecture/discussion, small group instruction, indi- 
vidual student-teacher tutorial, self-study with traditional textbook or textbook-derived 
materials, and self-study with programmed instruction. Recently, these traditional tech- 
nologies have been supplemented by radio or television instruction within the classroom, 
more sophisticated audio-visual equipment, and computers. These latter teaching-learning 
processes are the ones that are more "technological" but the term "technology" may refer 
to all forms of the educational process. 

The interaction of inputs and process determine educational costs. Ideally, educational 
managers should be able to design the instruction/learning system by considering alterna- 
tive inputs and processes simultaneously. However, the reality is that in most developing 
nations serious limitations exist in terms of the availability and quality of inputs and over 
the range of practical and affordable technologies (Thiagarajan, 1984; Cummings, 1986). 



FIGURE ONE 



MAJOR FACTORS IN THE EDUCATION PRODUCTION PROCESS 
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EFFECTS 

OUTPUTS OUTCOMES 

Cognitive Achievement Employment 

Improved Manual Skills Earnings 

Attitudinal Changes Status 

Behavioral Changes AtUludinal Changes 

Behavioral Changes 



The predominance of teacher-centered lecture/discussion as the means of educational 
technology is neither an accident nor a result of unfettered choice. Rather, this mode of 
classroom instruction has emerged because, first, many educational budgets must allocate 
80% or moreof expenditures to teacher salaries (with a substantial portion of the remainder 
used for system adminisuation) and, second, because teacher-student ratios are such that 
a lecture format is seen by most teachers as the only means by which the teacher can deal 
(in terms of instruction and discipline) with the large number of students for whom they are 
responsible. While most teacher uaining systems advocate greater use of small group and 
individual instruction, the teacher's own classroom experiences as a studentand the reality 
of classroom management demands often dictate against all but the most highly structured, 
•eacher-centered forms of classroom organizauon. Also, given the high rate of incidence 
of unqualified or underqualified teachers in some educauonal systems, reform of the 
teacher-centered instructional process, which may threaten the security of the existing 
teacher corps, is unlikely to occur without substanUal external pressure. 

A danger of the economic production metaphor is that it tends to imply that the 
technology used is rigid and constant and that the inputs are standardized and independent. 
Because the education process deals with human factors, all of these implications are 
unfulfilled to some degree. It is not just that variety (perhaps extensive vanety) exists 
among the inputs of teachers, students, schools, and materials, and the way they are com- 
bined" the individual human and material inputs also may vary over ume. The moUvation 
and effort of the teacher rmy Huctuate day to day or even within a given day; the 
attenUveness and effon of students is a notoriously variable commodity. The mtcrdepend- 
ence of the variables is indicated by the fact that one of the explicit responsibilities given 
to teachers is to monitor and motivate the behavior of the students; a similar indicauon of 
interdependence of inputs is the finding, common to the research literature, that student 
peer infiuences have a substantial moderating impact on student behavior and accomplish- 
ment (Winkler 1975; Webb, 1982; and Nielsen, 1982). At least in part, these interaction 
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effect can be controlled for by the introducUon of interaction terms in the quantitative 
speciricauon of an education production function. 

Thus, this abstracUon of reality from the conceptual form of production must be 
ra:ogmzed but it does not destroy the value of the production metaphor for understanding 
educauonal behavior. For example, in some classrooms it will be the practice of the teacher 
to spend extra time with the slower learning students while allowing the faster students to 
work on their own with textbooks or other materials. This is a decision that potentially is 
supportable from the economic theory of production. The teacher is operating on the belief 
that the marginal value of a unit of his or her time is more valuable to the slower learning 
student than to Lhe more advanced student. Even if the advanced students would learn more 
from the teacher than from study on their own. the greatest relative productivity advantage 
lies from combining the teacher input with the inputs of the disadvantaged students. 

Unfortunately, detailed microeducational analyses to support such teacher decision- 
making are almost never done. Classroom observation studies are the only practical means 
for developing the appropriate data for such analyses and such ecological studies of the 
school are time and labor intensive and suffer from their own setof serious methodological 
imitations as well as the obvious question as to generalizability. However, if classroom- 
level studies face serious limitations, as to method and relevance, the same can be said for 
the survey approach to analysis of educational production and efficiency The survey 
collection of data on inputs and outputs from a large number of schools assumes implicitly 
that the individual classrooms are using the same instructional technology when classroom 
observation studies often suggest this simply is not so (Thomas and Kemmerer 1983) The 
survey approach may be more acceptable in developing nations where, as noted above 
variation in formal classroom organization and process often is more constrained How- 
ever, in this setting the problem of proper specification and measurement of the variables 
and internal variation within a defined variable, may be even more of a problem than in a 
developed nation setting. 

To this point, the discussion of educational production has emphasized only inputs and 
processes (technologies). These two factors will determine the cost of education since total 
cost IS equal to input unit costs (cost per teacher-year or textbook) multiplied by input 
quantity (number of teachers or of textbooks). 

One of the major confusions concerning the efficiency concept is the belief that it is 
synonymous to lower costs. In a case where excessive expenditures and waste exist the two 
may be achievable simultaneously. However, where more cosUy inputs exist that have 
proportionally even higher productivity, the achievement of efficiency could be used to 
jusufy greater unit costs. In every case, cost consideraUons are only one part of the 
efficiency calculation. 

As indicated earlier in figure one. the effects side of the efficiency equation involves 
both outputs and outcomes. Outputs are the direct and immediate effects of the 
educational process. They include cognitive achievement, manual skill development. 
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allitudinal changes, and behavioral changes. In aggregate measurement one is concerned 
not just with measures of the central tendency but also distributive parameters. The latter 
are used in judging the equity or fairness of the educational system. Comparisons of such 
measures as student means and standard deviaUons among socioeconomic, ethnic, loca- 
uonal, or other classifications and between the gender groups is another method used to 
judge whether education has an ameliorating, neutral, or reinforcing effect on initial social 
disadvantages of given groups. 

Ideally, cognitive, manipulative, attitudinal, and behavioral measures of outputs should 
be highly'differentiated by specific subject areas, skills, attitudes, and behaviors. The 
common situaUon is that these measures are available only for certain general cogniUve 
areas (verbal and mathematics) and often not available at all in the areas of manual skills, 
attitudes, and behaviors. 

In addiuon to the differences noted above, outputs tend to be less subjectively measured 
than arc outcomes. The types of cogniuon, manual dexterity, attitudes, and behaviors 
purposefully promoted by the school are generally a product of governmental if not public 
consensus. The larger social outputs are more controversial both because they are less 
directed in their production and because they often involve the manifestation of unauthor- 
ized if not unacceptable views and behaviors. 

The difference in the degree of subjectivity is not absolute since considerable debate can 
and docs exist about what the school produces, whether the production is "purposeful," and 
how to value it. The tendency to value educaUonal outputs in terms of how they promote 
desired economic development outcomes has been one of the most controversial areas. 
Whether these development outcomes occur within market or statist economic systems, 
there is a legitimate question of whether other outcomes of education have not been unduly 
neglected in favor of this single indicator of educational effectiveness. 

Outputs, when compared to edu :ational costs, can be used in measuring internal 
efnciency is a measure of how well the educational insUtuUon or system achieves its stated 
goals - it is calculated by the ratio of output to cost. If both output and cost can be quantified 
in monetary terms a beneHt/cost ratio can be derived. To be efficient Uie benefits must 
exceed the cost (i.e., the benefit/cost ratio must be greater than 1.0). In comparing 
educational activities in the absence of practical budget constraints, the acuvity with the 
higher ratio of benefits to cost is preferred. 

If the effects of an activity cannot be stated in monetary terms, it is possible lo derive a 
cost-efu ctiveness ratio; however, the measure of effectiveness must sUll be quantifiable 
(even if only in an ordinal form). For example, a study might show that an additional five 
dollar pcr-student expenditure on instructional materials wi?l increase measured achieve- 
ment by ten percent while a similar expenditure on instructional radio increases achieve- 
ment by only seven percent In this example, the instructional materials altemauve would 
be the more cost-effective. 
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One weakness in many educational innovation projects is that the efficiency comparison 
IS made only between the individual innovaUon (addiUonal educaf ^onal materials or radio 
mstrucuon, for example) and the traditional classroom pracuce (lecture/discussion without 
mstructional support materials or radio instrucUon). In the ex-.-nple presented above, both 
systems might be judged cost-effccUve compared to the tradiuonal classroom; however 
the relatively less cost-effective radio alternative might be selected for implementation if 
It were the only mstructional alternative to the traditional classroom for which a cost- 
effectiveness comparison were made. Thus, the usefulness of cost-effectiveness analysis 
may be seen as a funcUon of the analyst's thoroughness in selecting options for evaluation. 

A second methodological problem is that some cost-effecUveness comparisons fail to 
consider the consequences of expending an equivalent addiUonal sum per-student on the 
traditional classroom alternative. Legitimately, no objecUve comparison of cost-effective- 
ness can be made unless either the cost or effecU veness standard is fixed. For example, one 
can compare the efficiency of the tradiuonal classroom with an instructional innovation if 
one has the same cost for both; in this case the difference in measured effecUveness alone 
will determine the more efficient alternative. Similarly, if the effecUveness standard is 
fixed (e.g., a five percent gain in measured achievement), it is possible to compare the costs 
to see which insUiicUonal system requires the least expense to generate Uie idenufied level 
of effectiveness. 

However , if neither costs nor effecUveness can be fixed for the two altcmaUves, it is not 
possible to use the mechanisUc criterion of cost-effectiveness. Rather, a cost-utility 
comparison must be made. An example would be where there are two innovaUons with data 
available as indicated below: 



Innovation A 
Innovation B 



Additional Cost 
Per Student 

SIO 

$15 



Average Percentage 
Increase in Achievement 

7% 
10% 



In this example, it would be fallacious to consider the relaUve cost-effecUveness ratios 
of S 10/7% and $ 1 5/10%. The fallacy exists because the fracUons do not contain a common 
numeric (unit of expression in quanUtative terms) and because one cannot assume that the 
cardinal valueofSl per student iseitherequivalentor consistently proportional to the value 
of a 1% increase in achievement 

Judgment in such a case must be made based on the subjecUve valuation that the 
decisionmaker assigns to the measures of costs versus the measures of effecUveness. One 
person may feel that it is worth the additional S5 per student to gain another 3% in 
achievement and thus would favor InnovaUon B; a second person might disagree and feci 
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that Innovation A is preferred and the addiUonal cost of $5 per student is not justified to 
produce "only" another three percentage point gain in achievement. Of course, it is also 
possible that a third person may feci both innovations are too expensive and would oppose 
the adoption of either one. The argument would be that the subjective value of increased 
achievement is simply not worth the additional expenditure. 

When one considers the fiscal effect of mulUplying a small increase per student times 
all of the students in an educational system, it is easier to understand why educaUonal 
innovations have such a history of disappointment in terms of system-wide adoption or 
adaptauon within developing nations. The advocates of specific innovations, in addition 
to being guilty of certain evangelical excesses in ignoring other innovative altemaUves. 
often fail to collect the data or institute the social marketing practices (Middleton, 1986)— 
including incentives— that will convince parents, practiUoners, and policymakers that the 
positive effects of the innovation are worth the financial expense (and the non-monetary 
costs that may be incurred in terms of the disrupUon of traditional classroom and bureau- 
cratic practices). 

The final form of efficiency analysis is least-cost analysis. It involves the lowest level 
of conceptual sophistication of any of the analytical models for measuring educaUonal 
efficiency. It assumes that the desired outputs are fixed (but not necessarily quantifiable) 
and requires only that evidence be presented that the proposed means of producing the 
outputs is the least costly alternative available. The most common use of least-cost analysis 
is in the determination of the feasibility of project designs. In such a situation, a judgment 
must be made that the probable effects of tlie project will justify the educational 
intervention and the probable costs of the intervention are the least expensive means of 
producing the desired effects. 

All of the approaches to efficiency evaluation mentioned here are generic to project or 
program analysis and are not limited to the evaluation of educational activities.* The 
application of these approaches to the appraisal or evaluation of educational activities has 
been more controversial than in such areas as transportation or infrastrucuve development. 
Health and population activities are an exception in that efficiency analysis has been at least 
as controversial there as in education. Education, health, and population activities share an 
immediacy in their effect on human lives and an inherent subjectivity in terms of external 
and coUecUve judgments of their benefits and costs. The controversy has been aggravated 
by a tendency of some efficiency proponents to misrepresent the degree of objectivity 
implied by the use of efficiency criteria such as benefit/cost ratios and cost-effectiveness 
comparisons. The fact remains that as long as educational demands exceed available 
resources, rationing of educational opportunity in access and quality terms will be required. 
Increased efficiency, therefore, will remain the only means of expanding access and/or 



* More extended discussion ofthese issues may be found inDonohue, 1980; Levin, 1983; 
Ray. 1984; and Woo and Sanders, 1986. 



13 



2-1 



Chapter! 



quality without new resources and the best means of assuring that new resources that are 
made available are used to maximize their impact on access opportunity and quality 
enhancement The efficiency analyst can help provide the most current and accurate 
information obtainable to assist the decisionmaker in promoting access, equity, and quality 
enhancement goals. 

A special aspect of the efficiency controversy is the difficulty of shifting from the 
analysis of mtemal efficiency to the analysis of external efficiency. While disagreement 
may and does exist over the degree of determinacy of ed^^cational inputs for educaUonal 
outputs, the difficulty of relaUng the immediate effects to the investments made in 
educauon are much less than in attempUng to relate the more distant educational outcomes 
The long term effecLs that one normally considers as educational outcomes are such things 
as hfeume earnings, the probability of employ ment, occupational attainment, social status 
poliucal participation, consumption and savings patterns, and a variety of attitudes and 
behaviors. 

Two main difficulties exist in relating educational inputs to such educational outcomes 
(that IS, in the measurement of external efficiency). The first difficulty is determinacy 
(imputing the causal effect of education) and the second is discounting for time preferences 
The latter problem exists because educational expenditures that are investments in future 
outcomes require an immediate financial sacrifice in return for a future benefit that may be 
delayed for a considerable time. 

The concept of Ume preference is well established in behavioral psychology and eco- 
nomics. Two primary reasons are given for a preference for immediate versus postponed 
benefits: the first is the risk that a benefit delayed may not be received or not received in 
full and the second is the demonstrated preference ot individuals for immediate over 
delayed (but otherwise assured) consumption. Mortality, changes in educational qualifi- 
cation requu-emen:^and other changes in the labor market relating to salaries and job 
ser uritv make it exceedingly difficult to predict the future earnings, employment probabil- 
ity, or status for any given level and type of education graduate. 

When dealing only with financial outcomes such as earnings it is possible, through 

discounting,tocompare monetary valuesacross time. Aunitof currency at any future time 
may be equated to a current unit of currency values by the following formula: 



FV 



PV = 



(1 + i)' 



where: PV 
FV 
i 

and t 



present value 
future value 
rate of discount; 
number of time periods 
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Thus, if the future value is $100 and the rate of discount is .10 (10 percent) per time 
period and the future value is received five time periods into the future the formula 
becomes: 

$100 

PV=: =$62.09 

(1 + .10)^ 

Thus, $100 received in five years is equivalent to receiving $62.09 today if the rate of 
discount is ten percent. The rate of discount incorporates the effect of anticipated inflation 
plus other calculable risks involved in postponing receipt of funds. Because of psychologi- 
cal and other differences among persons, individuals' subjective rates of discount may 
differ substantially. The present value of benefit/cost then is exactly the same criteria as 
the normal benefit/cost criteria but with the important exception that the values of benefits 
and costs have be^cn adjusted by discounting to take into account when the benefits and costs 
occur over time. An alternative to the present value of benefit/cost criteria is the rate of 
return approach that will be discussed in detail in the later section on efficiency criteria. 
The present /alue formula may be modified to calculate the sum of the present values of 
a series of different future values that occur over a number of time periods (for example, 
expected annual earnings over a period of years): 

N FV^ 

PV = Z (1 + i)^ 
t= 1 

where n = the total number of time periods. 

While the discounting process is apparently objective, its mechanistic nature disguises 
the problem of obtaining the appropriate estimates of the future values and the difficulty 
of selecting the appropriate rate of discount. The future earnings for a certain type of 
educational graduate may be forecast based on current earnings patterns and expected labor 
market changes — this is a process fraught with the opportunity for substantial error. Also, 
fifty years ago, the rate of discount was considered relatively uncontrovcrsial because 
lending rates for *'riskless" government bonds (the normal basis for determining the 
opportunity cost of the lime delay) were relatively standardized and tended to be stable over 
time. In the current capi* a\ markets of most nations a plentitude of rates may exist without 
clear criteria (such as \ arying maturities or risks) to justify selection of a single discount 
rate and, more importantly, the fiuctuation in rates over time may be expected to be much 
more substantial due to variations in the expected rates of inflation. 
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These difficulties make the calculation of present values of educational outputs difficult 
but still feasible if done on a relatively frequent basis and if one can avoid making 
substantial fixed investments on the basis of present values that may change significantly 
over time. The latter caveat is important but often ignored in educational planning 
decisions. Where possible, educational investments should be of a type where fixed facili- 
ties and equipment are avoided or minimized and, again, where possible, subject to 
alternative uses if future conditions no longer justify continuing a project or program. 

The areas of secondary and post-secondary vocational training or technical education 
are excellent examples of where this logic can be applied. The demand for vocational skills 
may fluctuate greatly over time and, within a single economy, specific skills may be subject 
to saturation in supply in a relatively short period of time. For example, if there is a need 
to produce a total of 1 ,000 electricians over the next five years one might create a training 
program that would produce 200 graduates per year. The problem is that at the end of the 
fifth year the demand for electricians may be satisfied but the training program will still 
exist. Educational systems have had little success in closing programs once they are 
initiated. Ideally, the original program plan should have presented efficiency data to justify 
the production of the 1 ,000 graduates but also should have provided an analysis of how the 
program could be phased down, converted, or terminated once the justified number of 
graduates were produced. 

An important reason for the growing emphasis on the use of industrial sites for training 
activities is that, in addition to providing access to more current technology, the main cost 
of equipment is for the purposes of production, not training. Therefore, the efficiency 
analysis requires only that there be sufficient benefits to justify the proportion of equipment 
and facilities costs allocatable to the training activity rather than the total of such costs. 

The preceding discussion has concentrated on the issue of production. Because both 
outputs and outcomes of education are multiple, and even some individual outputs and 
outcomes must be valued subjectively, the economic concept of utility must be discussed. 
Consumer utility may be defined as the pleasure or satisfaction a consumer expects to 
receive from consumption of a product or service. When the "consumer" is in fact a 
bureaucrat or other policymaker, their utility ideally should be inclusive of judgments 
about the probability of consumer satisfaction on thepartof the individuals affected by their 
judgment. For example, an educational planner's decision will be based upon his or her 
personal utility but also on the degree of responsiveness of the planner to perceptions of the 
utility of the parents, teachers, students, and others affected by the decisions made. 

In the simplest case, where only two outputs exist, utility maximbation will be achieved 
by considering the value of the two outputs and (he resource constraint on output 
production. While mathematically the problem of maximization of utility increases as the 
number of outputchoices is expanded, the fact is that the human mind (and now, computers) 
can handle the optimization process quickly. The main constraints the educational 
decisionmaker faces arc the paucity of knowledge about alternative costs, about the nature 
of relati vnships among outputs, and about the tim^: preferences of tiiose affected by the 
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decisions made. Some outputs— such as verbal ability and certain forms of disciplinary 
behavior or obedient attitudes— may be joint outputs. This means that the process of 
producing one output can produce the other output at no additional cost. Other outputs 
may be mutually exclusive at the margin. That is, one can produce more of either output 
but not more of both. An example would be that one may not be able to produce, greater 
achievemenf by the most advantaged students and increase achievement equality or the 
class at the same time. 

The task of educational managers (in fact, of ail managers) is to understand the 
production process well enough to be able to identify which outputs are independent, which 
are joint outputs, and which are -nutually exclusive outputs. Then, the educaUonal utility 
decision requires combinauon ... this knowledge of the production process with an 
understanding of the appropriate values to be assigned to the outputs so that a decision can 
be made that will maximize the utility to be derived from the mix of outputs that are to be 
produced. 

There is an unfortunate tendency for politicians and even some senior educational 
administrators to act as if the educational production process can be expanded (in terms of 
the number of outputs and/or the amount of the individual outputs produced) without 
providing new resources or incurring any sacrifice in existing output production. Implic- 
itly, they are assuming that the current educational process is inefficient (probably true) and 
can be changed by administrative fiat (probably false). Unfortunately, even if the 
administrators were fully correct, administraUve decrees rarely produce new outputs 
without reducing others. The demand for new or better ourputs may be achieved by 
sacrificing some existing outputs that could be more highly valued. In addiuon, the new 
demands may overburden the process (especially the teachers class management skills) in 
sucha manner that overall producUon of outputs is reduced. Too often ignored, both m such 
a manner that overall production of outputs is reduced. Too often ignored, both m 
economic theory and administrative procedures, is the fact that those direcung the 
educational production process at the classroom, school, and even system level are 
individuals who rarely have received management training concomitant with their man- 
agement responsibilities. 

A final item of terminology needs to be reviewed before proceeding from this discussion 
of the concepts of producUon and utility to their application to effectiveness and efficiency 
analysis in schools and school systems. A critically important term is that of the"margm;' 
Frequent references refer to marginal cost or marginal productivity and it is common for 
economists to say that a certain condition (e.g., equilibrium or efficiency) exists "at the 
margin." The concept of margin is taken from the calculus and refers to a single 
incremental unit. For example, and marginal productivity of teacher ume could refer to the 
increase in prcducUvity that wouid result from one more hour of teacher effort. Similariy 
the marginal cost of teacher time could be defined as the expense of using the additional 
hour of teacher time. In theory, marginal units are assumed to be extrmely small; in 
practice, one often is forced to work with units of substantial size (a person-month or 
pcrson-y car , for example). One can even consider the idea of a marginal school or marginal 
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university if one is analyzing the effect of adding an addiuonal instituuon to an existing 
system. 

Five limitations exist in regard to application of basic producuvity and cost relaUonships 
to education: 



1. multiple inputs that must be determined simultaneously; 

2. multiple output/outcome measures of productivity; 

3. lack of information on costs and productivity; 

4. fixed input quantities or relationships; and 

5. variable input quality. 

The problem of multiple inputs is one faced in almost all production situations but poses 
special problems in education. While economic theory stipulates productivity relation- 
ships under ceteris paribus conditions, the educational decisionmaker must determine the 
mix of inputs simultaneously. Teacher quality and quantity, availability and use of 
materials, equipment and facilities, and means for motivating student, parent, and commu- 
nity effort are some of the major input categories with which the decisionmaker must deal. 
One reason for the conservatism of educational systems relative to instructional change is 
that the decisionmaker always has to justify any new input mix when, in fact, there are few 
data or experiences to support the purported effectiveness of the new input types or 
quantities. In addition, economic theory presents the productivity concepts in terms of a 
given technology. If the new input mix also involves a change in technology, it will be even 
more difficult to justify the instructional change on the basis of a priori quantitative data. 

The second problem the educational decisionmaker faces is the valuation of marginal 
product. In addition to the basic problem of value judgment, .le decisionmaker must 
Identify and value the effect of the individual inputs on multiple outputs and outcomes. As 
discussed above in the description of utility analysis, multiple products can be dealt with 
but they add complexity to the analysis and heighten the implicit subjectivity of the 
valuation process. The decisionmakers need to know both the cost of inputs and the 
relationship (independent, jointly produced, or mutually exclusive) among the inputs and 
the mix of outputs and outcomes. In addition they must be able to assign a value to 
alternative output/outcome mixes. Obviously, most educational production decisions are 
made without all of this information; the goal of efficiency advocates is to increase the 
amount, quality , and timeliness of such information and tomake the valuation process more 
explicit 

The third constraint on educational decisionmaking about production is the availability 
of information. Given the quantity of educational research of the last thirty years it is 
surprising how little is "known," let alone how litUe can be "proven" concerning educa- 
tional production and efficiency. The next section of this monograph will discuss the 
various individual inputs, processes, outputs, and outcomes that commonly are proposed 
for education. Each will be reviewed in terms of what research has revealed, what 
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deductive logic and experience can tell decisionmakers, and what can be done to increase 
the informational base for efficiency decisions. 

The fourth specific constraint of educational decisionmaking concerning efficiency is 
the fixed nature of relationships that exist within the educational production process. These 
rigidities are not always technologically determined but rather are often a product of 
tradition, law, regulation, or contractual agreement. The most dominant of these rigidities 
is the central role for the teacher. The teacher's dominance in the classroom is an 
interesting example of tradition becoming institutionalized by law, regulation, and con- 
tract. Further, because of the low level of resources normally available for the classroom 
instructional budget, there is little ability in the poorer countries even to provide significant 
complementary inputs to reinforce the teachers' effectiveness, let alone to consider replac- 
ing the teacher as the major input 

The fifth major limitation on the use of economic production and cost concepts in 
educational management is the variability in the nature of the inputs. The major cause of 
this variability is the need to conduct management decisionmaking at an excessively high 
level of aggregation (and often at a physically distant level of administration). The 
"teacher" input is an example of a variable that often contains substantial internal variation; 
in such a case the modal or mean characteristics of a group of teachers may not be a useful 
base for decisionmaking because of the large range and substantial variation that exist 
around these measures of central tendency. Even if one divides the excessively aggregated 
concept of the teacher input into its component parts — subject knowledge, experience, 
pedagogical skills, motivation, attitudes, and behavior — the problem of internal variation 
within the multiple teacher-input definitions still may be considerable. 

This problem is at the center of the long-standing debate over survey versus observa- 
tional collection of data with which to analyze education production relationships. While 
observational techniques provide more depth and detail in terms of measurement of the 
variables and their interaction, the observational approach itself has three major methodo- 
logical disadvantages. First, observational measurement techniques are still in the process 
of development and controversy still exists over the specification and measurement of 
educational variables at the classroom and school level. Second, there is an unavoidable 
and explicit acceptance of subjectivity and variability in the measurement of inputs. An 
observer measuring time-on-task of students is forced constantly to make judgments of 
student behaviors as to whether certain actions arc learning relevant or not. In addition, 
there is the fact that observed values will differ not just among observers but that the values 
of inputs assigned by a single observer can vary from situation to situation depending on 
the observer's attcntiveness and diligence. And third, because classroom and school 
observational studies involve substantial cost in time and money, this methodology allows 
results of only limited immediate generaiizability because the research budget rarely 
allows a statistically representative coverage of classroom or school settings. 

The weaknesses of the observational methodology for analysis of educational produc- 
tion are no greater than those that exist for the survey methodology; however, because the 
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weaknesses are more obvious (and the survey approach has tradition as an advantage) the 
observational methodology has been underutilized. The pointmust be made, however, that 
some educational production issues are researchable only by ethnographic methods 
including classroom observation. A major need is for economists and others interested in 
educational production relationships to develop a consensus as to the situations in which 
the survey or ethnographic approaches have a comparative advantage. This consensus 
could then serve as a basis for design of both educational management information systems 
and for a more comprehensive research agenda for the study of educational production and 
efficiency. 

The limitations discussed here must be understood within the context of a more far- 
reaching limitation. Education's conservatism toward the application of efficiency 
concepts is simply a special case of the general administrative conservatism towards all 
change. Unlike the case in the more progressive parts of the private sector, bureaucracies 
rarely offer incentives for managers to engage in change or experimentation and educa- 
tional bureaucracies in particular often exert specific sanctions against innovation. In this 
context itis easy to understand why improvements do not occur even if understanding fails 
to reduce one's frustration with the process. 

In this section an attempt has been made to accomplish two objectives: 

1. familiarization of the non-economists with the basic economic concepts that 
support the measurement and analysis of educational efficiency, and 

2. development of a standard set of terminology for efficiency analysis so as to 
minimize semantic confusion in the succeeding discussion. 

Ideally, the purpose of a presentation such as this should be to simplify and clarify; to 
many non-econom ists the initial reaction to the preceding discussion may be that the whole 
area of educational production relationships now appears more complicated than before. 

The reason for this is that the presentation of education as an input-output process 
analogous to other technical pi xluction relationships is simply wrong. While the input- 
output model may have great va. ue as a metaphor to help the uninitiated gain some basic 
appreciation of educational production, understanding educational production relation- 
ships requires that one move to more complicated economic models (involving the 
complex relationships among multiple inputs, the consideration of variable technologies, 
and the subjective valuation of educational outputs and outcomes). But to be of any 
possible policy value, the economic models must be understood to provide only a 
framework within which behavioral psychology, pedagogy, administrative and manage- 
ment science, anthropology, political science, and information theory all must play 
important roles. Finally, one is left with the realization that all educational decisionmak- 
ing will take place without optimal information and will be performed by individuals who 
lack the ideal mix of personal and professional skills and experience. 
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But in this regard education is no different from the other social services; the point is 
decisions must be made and will be made. The function of the efficiency analyst is to 
improve both the decisionmaker and the decisionmaking process. Improved, not ideal, 
decisions are the only realistic and attainable goal. 

Within this more restrained statement of the goal of education production analysis, one 
must face the fact that even improvement can never be certain. Production analysis for 
education remains limited by what is understood of the production relationships and what 
data can be generated (in a cost-effective manner) to support decisionmaking. In the next 
section of the monograph, the purpose shall be to present what is known (or more correctly 
what analysts think is known) about educational production and efficiency. 

While analysts should be modest about their level of conceptual or factual knowledge, 
they have no choice but to be relatively immodest in promoting the use of their knowledge 
by practitioners, planners, administrators, and policymakers. The cost, in financial and 
human terms, of erroneous educational policies is simply too great. The educational de- 
cisionmakers may choose to distrust or ignore the analysts' recommendations but it is 
crucial tiiat the decisionmakers at least be exposed to tiiem and allowed to consider them. 
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INDICATORS OF EFFECTIVENESS IN EDUCATIONAL 
PRODUCTION: INPUTS AND PROCESSES 

In this chapter, educational effectiveness indicators will be discussed at the first two 
stages of educational production: inputs and processes. It should be repeated here that 
whUe the discussion will continue to focus on examples from primary and secondary 
education.theconccplspresentedandinterpretationsmadeoftenwillbeequallyapphcable 

to pre-primary education, vocational and technical schools or programs, post-secondary 
education, to pre-service and in-service teacher training programs, and to nonformal 
education. 

Since efficiency is definable only in terms of both inputs and outputs, some question 
might be raised as to the propriety of discussing input and process measures as indicators 
of effecUveness They are included here since, because the preferred output and outcome 
measures of educational effects are frequently absent, analysis often are forced to attemp 
to evaluate a school or program only on the basis of inpu^ and processes As PO'nted out 
bv Chapman and Windham (1986). school "quality" definiUons are as I'^ely ^o re er to 
inputs and processes as they are to outputs or outcom^ (e.g.. '^^'■1':'^^^^^^^^^ 
Heyneman and Loxley. 1983A. 1983B; Behrman and Birdsall. „ Fuller. 1985 ^nd 
lEES ]985).Thus.itseemsapi.ropriatetoincludeinthepresentdiscussionarev.ewofthe 
«s that shoild help determine not only the costs of the school but its eventual 
effectiveness in achieving desired outputs and outcomes. 

A second question might be raised as to the exclusion of home and community 
environmentvariables from ihisdiscussion.Thesecontextualdeterminants are recognized 

as having critical importance and it is understood U.at educaUonal planners and admim - 
is need to evidence a greater sensiUvity to the effects of the home and community 
cotext (Selowsky. 1980; Mercy and Steelman. 1982; BirdsaU and Cochrane. 1982, and 
Johnstone and Jiyono. 1983). However, the purpose of this discussion is to examine the 
variables that are within the control or infiuence of the school aammistrator or plann r 
While it is not possible in the short run to alter parental education or earnings or to alter 
significantly the availability of educational and cultural opporluniUes withm a given 
community!one process v^iable that will be discussed here is the school's success m 
motivating parental and community involvement. 



I. INPUT INDICATORS 
A. Teacher Characteristics 



The tendency to judge Uie quality of a school or other educational institution by the cost. 
quantU anS quality of i^ inp Jts is not limited to developing nations. Often paren^ 
student administrators, and analysts have no other measures from which to make an 
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evaluation Also, since the inputs are within the direct control or inOuence of the 
educanonal authority, it has always made a certain intuitive sense to fZusm'nZ on Te 
aspects of the school that can be affected by administrative personnel 

ThJIlLT'' "^f/^r"^^ f '^''^ '"P"' the teacher's characteristics 
Theteacheras the ocusof classroom instru^ 

all cultures and has been institutionalized in most curricula and forms of ch^ rim 
organ^auon. Also, as was discussed earlier, many developing atir^flce suchTal 
onstxamts and alternative priorities that it is extremely difficult to op fo oSier t^^i 
teacher-centered curnculum: after the teachers' salaries are paid there are ew funcLreft in 
the educauon budget for alternative or even teacher-suppoZethods of instruction 

A consideration that often is ignored in this debate, especially by those who promote 

V rtr r^LXr^^^ r"""^ '"r^^'^^"' employment LTes a 

^Z^l ■ and social purposes for any government (Illich. 1970). Even where 

/Zl . ^M- government. Even those who advocate less radical reforms 

e e such '""''"^ with alternative learning u^hnZ 

hZv7^t P^grammed .nstruction or interactive radio) often will find Lmse^S 
blocked because parents and government feel that improved teacher quality TZmoZ 
visible and reliable means of school improvement ^ 

oft'e^^htrTulmyt''"^^^^^ 

formal educational attainment 

teacher training attainment 

ag^experience 

attrition/turnover 

specialization 

ethnic/nationality 

subject mastery 

verbal ability 

attitudes 

teacher availability measures 

for'^^i^^a tLtTrn"''" ''"^''^ ^''^^^^^ -dividual has 

for being a teacher. The amount and quality of both academic education and teacher 
uaining are assumed to be positively correlated with the teacher's knowledge and w!^ U.e 
teacher s ability to impart that knowledge to students. 

1.vhT V ^' ^""^ "^"^^ '"^''^''^'^ of education or irainingand the 

level of highest attainment. Sometimes, these indicator, are expressed in terms of U^e 
government's or other authority's standards for qualification: 
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qualified — possessing the academic and teacher training altainmeniappropriaie 
to the assigned level and type of teaching. 

underqualified — possessing the academic but not the teacher training attainment ap- 
propriate to the level of assignment; 

unqualified — possessing neither the academic nor the teacher training attainment 
appropriate to the level of assignment 

The use of the qualifications measure for inter-counlry comparisons can be misleading 
since each counliy is free to establish its own standards. These standards often have as 
much (if not more) to do with the status of teacher supply and demand as with any objective 
standard of the appropriateness of education and training attainment to teacher assign- 
ments. Still, within a country or among countries with comparable standards and 
education/training systems, one can use the percentage distribution of teachers across the 
three standards of qualification as an indicator of input quality. 

A second point of significance about the education and teacher training credentials of 
teachers is that most government and private pay systems reward higher levels of 
attainment. Thus, even if the assumption is correct that higher levels of attainment promote 
better instruction, the concomitant effect of teacher educational attainment on educational 
costs means that to be efficient, one must assume or be assured that the marginal cost of 
higher attainment qualifications is offset by the increased marginal output of the class- 
rooms or schools in which the "more qualified" teachers are employed. In terms of 
government policy there often is an immediate and recurrent cost impact from upgrading 
teacher qualifications. The immediate effect comes from instituting or expanding the 
academic and teacher training programs necessary to produce a greater number of teachers 
with higher qualifications. The effect on recurrent cost is a result of having to pay greater 
annual salaries to the teachers once they attain higher qualifications. 

A common phenomenon during a period of educational expansion is that there is a 
compounding effect from the interaction of the pay system attempting to absorb the impacts 
of both more teachers and higher teacher qualifications for old and new teachers. Without 
proper consideration of budgetary impacts, such programs can create general fiscal 
problems and pose serious opportunity costs in terms of other social and development 
programs. 

The teacher characteristic of age/experience is equally controversial. The age of the 
teacher is an indicator used as a proxy for either emotional maturity or experience when 
these characteristics cannot be measured directly. Also, in many cultures, the age of a 
teacher is an important determinant of the authority and respect that will be granted by 
students, parents, and community. Without this authority and respect, the education and 
training attainments of a teacher may be irrelevant 
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Even where direct measures of experience are possible there may be a substantial gap 
between conceptualization and specification. As a concept, experience implies the 
embodiment of skills that occurs over time from the formal and informal learning 
opportunities to which the teacher is exposed. However, the experience variable normally 
is specified in terms of the number of years the individual has been a teacher. The 
conceptualization and the specification undoubtedly are correlated but the degree of 
correlation is subject to debate; it varies from teacher to teacher and. more importanUy. 
varies within and among counu-ies based on the availability of the learning opportunities 
for teachers outside the on-the-job experiences that occur normally within a classroom. 

Like qualifications, age (as a proxy for experience) enters into some educational pay 
schemes as a determinant of salary. The most common specification of the age/experience 
variable is years -of-scrvice. This is defined as the number of ye^s from iniual employment 
to present employment (if continuous) or the sum of years of teaching if employment has 
been interrupted at any time. Often, pay systems combine the years-of-service concept 
with qualifications into a pay system that has separate pay "steps" for each level of 
qualification and. within the qualification level, individual pay steps based on years-of- 
servicc at that qualification level. 

Regardless of the form of instituting age and experience within the pay system, the 
bcnefit/costconsideration is the same as for qualifications. One must assume or be assured 
that the extra cost of having older, longer serving, or more experienced teachers is at least 
offset by the differential effect of these teacher characteristics on classroom and school 
outputs and outcomes. If not. there is then no educational justification for the pay system 
(although there may be significant social or political justifications). 

Inversely related to the experience concept is thech;u'acteristic of teacher attrition. The 
loss of teachers from the educational system through retirement or resignation can involve 
a loss of exactly those personal qualities that the pay incentives for age and experience were 
designed to promote. Thus, teacher atu-ition rates may be used as indicators of potential 
educational ineffectiveness at the classroom or school level; teacher turnover (based upon 
reassignment as well as retirement and resignation) is equally valid as a negative indicator 
of effectiveness. 

Interestingly, the atu-ition and turnover indicators are less clear when applied to 
efficiency issues. Since the teachers who retire or resign may be (and usually are) replaced 
by teachers who are younger, less experienced, and/or less qualified, teacher attrition 
normally is concomitant with a lower cost of instruction. Again the educational authority 
is faced with a judgment concerning costs and benefits. Are the reductions in cost from the 
change in teachers enough to offset the probable redirections in the value of school 
effectiveness? 

A more specific teacher qualification issue is the match of teacher specialization with 
the requirements for teachers. The most common problem, and one that occurs in both 
developed and developing nations, is the shortage of teachers trained in science and 
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mathematics. Two of the most serious errors of aggregation in the analysis of teacher 
supply and demand are: (1) to ignore teacher specialization and therefore assume thata bal- 
ance occurs when total teacher supply equals total teacher demand— in fact, a surplus of 
arts or social studies graduates does not solve the problem of a shortage of science and 
mathematics graduates; and (2) to ignore the geographic distribution of teachers by 
specialization— a national balance of teachersby specialization can disguise an urban over- 
supply of science and mathematics teachers and a rural under-supply. 

Similar issues exist relative to the supply of instructors for vocational skills, technical 
concepts and applications, foreign languages, and many of the undergraduate and advanced 
courses in iiigher education. Since most educational systems pursue the illogical course of 
undifferentiated pay by specialization, the reduced effectiveness of education provided by 
inappropriately trained teachers is never concomitant with financial savings. 

The reality, of course, is that attempts to remedy the maldistribution of teachers by 
specializations probably can occur only with an increase in the cost of teachers' services 
(and perhaps an increase in the cost of teacher training as more expensive teacher tramers. 
faciliues. equipment, and materials are required). It is important that all teacher character- 
istics, but especially this one. the appropriateness of subject specializations, be considered 
in terms of the impact on aggregate effectiveness and equity standards. 

A special characteristic of teachers that may represent a proxy for perceived educational 
quality or effectiveness is the teacher's ethnicity or nationality. In a multi-ethnic society 
students, parents, or others may identify positive or negative traits or behaviors with 
members of certain ethnic groups. While this is more likely to be an issue of perceptions 
than reality, planners and administrators must be aware of these community atutudes^ 
However, the exposure of students (and their families and communities) to members of 
other ethnic groups may be a purposeful determinant of the teacher assignment system. In 
any case, data on the distribution of teachers by ethnicity and location may be considered 
a valuable indicator of potential effectiveness in certain societies. 

Similarly, dependence on expatriate personnel as teachers may be interpreted either in 
a positive or'negative manner. As with ethnicity, Uhe foreign origin of a teacher may have 
a beneficial impact in terms of promoting understanding and tolerance. However, the use 
of expatriate personnel has two possible negative factors, one financial and the other 
pedagogical, that must be considered. The negative financial factor is the higher salary 
costs usually borne by the educational system for foreign teachers. An exception to this is 
when Uie expatriates are paid for or seconded from donor nations. The cost of those 
foreigners paid by the host govcrhment, however, also can involve a balance of payments 
issue since the foreign teacher may demand to be paid in a foreign currency or demand 
currency exchange privileges so they can convert part of their salary payment for 
repatriation to their home counuy. 

The negative pedagogical effects can occur because the expatriate may be unfamiliar 
with the local curriculum and the social and cultural context within which the curriculum 
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has been developed, nie tendency of expatriate teachers to ignore or underemphasize the 
local curriculum is increased in those cases where the expatriate does not expect to remain 
as a teacher in the host country for more than two or three years. An additional complaint 
often expressed about expatriate teachers is their failure to use the local language (or the 
local pronunciation) correctly and their inability to relate concepts to local history and ex- 
perience. 

Given that expatriate teachers cost more and may be less effective in some situations, 
why are they used? Obviously the explanation is that at the early stages of educational 
development the only means for meeting the demands for certain high-level and scarce 
teaching skills may be to employ expatriate teachers. This situation points out, however, 
the potentially powerful efficiency effects that can occur with indigenization of the 
teaching service. If quality sacrifices can be minimized, the benefits from the use of 
indigenous teachers can include lower salary costs, a reduced balance of payments effect, 
and the possibility of greater local and national relevance in the application of the 
curriculum. This discussion of the role and effect of expatriate teachers is illustrated most 
dramatically by the experience in die Yemen Arab Republic where, in 1982/83, only 
sixteen percent of primary teachers, seven percent of preparatory, and six percent of 
secondary teachers were Yemenis (lEES, 1986A). 

To this point, the characteristics that have been discussed are proxy indicators— they are 
not valuable in themselves but have importance only in that possession of these character- 
istics may be correlated with possession of specific abilities, knowledge, attitudes, and 
behaviors that are understood to promote the desired educational outputs and outcomes. 
Another set of indicators exists that relates more directly to these desired characteristics of 
teachers. 

Subject mastery is important in that it determines the extent of knowledge — of facts and 
skills— that the teacher can transfer. While obviously correlated with general attainment 
and subject specialization, subject mastery possessed by individual teachers will vary 
according to their own abilities, the effort they expended in knowledge acquisition, and the 
quality of training provided to them. Where subject mastery is lacking or inadequate, the 
teachers' knowledge can be supplemented by textbooks or educational support materials 
(Lockheed, et. al., 1987). 

Teacher verbal ability is a critical factor and, in some studies, the input of greatest 
determinacy in student achievement (see surveys such as Husen, et al., 1978; Avalos and 
Haddad, 1981; and Green, 1983). In the traditional classroom, the teacher's ability to 
communicate facts and concepts is the major facilitator of student learning. In developing 
nations that face a scarcity of other inputs such as textbooks and instructional support 
materials, the teacher's ability to communicate will be the major school-provided instruc- 
tional resource that will determine student acquisition of knowledge. Obviously, the total 
effectiveness of the teacher will be determined by the net effect of subject mastery and 
verba! ability. The cffcctof a high level of subject mastery can be diluted if a teacher has 
poor communication skills. Similarly, good communication skills are less valuable if a 
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teacher has Utile knowledge to impart to students. The success of teachers (and the 
appropriateness of the instructional program at teacher training institutions) can be 
predicted based primarily upon the level and complementarity of these two characteristics. 

A final important characteristic, and one that frequently is neglected in survey research, 
is the teacher' s attitude toward the classroom process. This would include specific attitudes 
toward children, the community, the school administration, their fellow teachers, and the 
occupation of teaching. Some of these attitudes will originate in the teachers' own expe- 
riences as students; some will be a product of their teacher training courses; and others will 
reflect general social and community attitudes. Over time, however, the most important 
teacher attitudes will be those that they develop or modify as a result of their own 
experiences as classroom teachers. The determinant of these attitudes is the configuration 
of positive and negative incentives that exist for different forms of expressed attitudes and 
behaviors. 

Measurement of attitudes is a difficult methodological task. Survey instruments suffer 
from a skepticism as to the willingness of teachers to record their true attitudes as opposed 
to the ones that they feci arc socially acceptable. * At the observational level, much more 
can be done in the measurement of attitudes even if the set of attitudes arc inferred froni 
teacher behavior rather than measured directly. Experience suggests that teachers' 
attitudes are a powerful force in determining their effort in their work and their empathy 
with students. The current interest in teacher incentives (going beyond salary and 
promotion consideration to an extensive number of monetary and nonmonetary influences; 
see Thiagarajan and Kemmercr, 1987) is based on the precept that teachers attitudes, and 
thus their behavior, can be modified by the actions of educational adminisuators, parents, 
and community officials. 

Four teacher availability measures are commonly used to express the number of teachers 
available relative to some other unit of educational input. These are the student/teacher 
ratio, the teacher/class ratio, the teacher per school ratio, and the teacher ir.structional hours 
per week. The student-teacher ratio is derived by dividing the number of students by the 
number of teachers (or, preferably , the full-time equivalent number when some teachers are 
employed on a part-time basis). This is sometimes expressed in the inverse form of teacher 
per student; while the teacher per student ratio has the advantage of indicating the average 
share of a teacher's time available to a student the more common student/teacher ratio is 
used here. 



* Even skeptics are surprised at times by the variety of attitudes that teachers can perceive 
as socially acceptable; in many nations the variety of types of socialization and the 
variation in degrees of social inculcation surprise those fam iliar with the more homoge- 
neous situation in developed nations. 
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Student-teacher ratios have been one of tlie least well understood measures used in 
educational effectiveness analysis (e.g.. see Haddad, 1978; Glass and Smith. 1978; Glass, 
et aL, 1982, and Hanushek, 1986). The assertion by many researchers has been that there 
is no proven advantage to small class size.* Given the rarity of small enrollments relative 
to teachers in most developing nations — except where these are concomitant with such 
forms of educational disadvantage as rurality, remote location, or multiple-class 
teaching responsibilities— the appropriate policy interpretation of these findings is less 
definite. Class size may be of great important in dealing with certain types of students and 
certain subject areas and should not be disregarded as an effectiveness variable. 

The teacher per class measure helps identify those situations where over- or under- 
utilization of teachers can occur because of an inability to match teachers to classes on a 
one-to-one basis as is common in most instructional systems (even where a teacher is a 
subject matter specialist teaching multiple classes over a day, the total for the day should 
still approximate one full-time equivalentclass per teacher). A ratio of more than one full- 
time or full-time equivalent teacher per class suggests that all teachers in the school or 
system are not fully utilized. This may be a result of requirements for additional non-class 
instructional activities, for outside class responsibilities (e.g., parental or community 
contacts), or simply a result of the need to provide a subject specialist to a school that is not 
sufficiently large to use the specialist full time. These explanations emphasize the fact that 
a ratio of more than one teacher per class is not proof of inefficiency; it simply requires a 
justification in terms of showing that the teacher's time is fully employed in other activities 
for the school or system or that the situation of low utilization cannot be remedied by 
alternative administrative arrangements (e.g.. school consolidations or employing teachers 
who work in multiple schools on a rotating basis). 

A teacher to class ratio of less than 1 .0 suggests that multiple-class teaching must exist. 
While this poses potential instructional and administrative hardships on the teacher, the 
situation may be inevitable in cases of teacher shortages or rural schools with small total 
enrollments. Again, inefficiency exists in such situations only where an instructionally and 
financially feasible alternative exists that would enhance educational outputs and outcome 
without an offsetting increase in costs. 

The teacher per school ratios are less readily intcrpr^ble because they require that one 
know the number of teachers required for a school. Some large schools will require 
multiple teachers for a single grade level; other schools that are incomplete will not have 
a full cycle of classes for their level of education; and, in rural areas, some schools may 
combine levels (normally primary and lower secondary) of education in a single school 
facility. The only certainty with this measure is that a ratio for an educational level that is 



* In fact, of 112 estimates reviewed by Hanushek (1986), only 23 had statistically 
significant effects of class size and only 9 of these had a positive sign. However, 
Hanushek himself warns of the danger of equating teacher/student ratios with class size. 
The warning is even more appropriate in developing nations where the ratio of grade six 
to grade one enrollments, for example, can be quite small. 
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lower than the number of grades in that level (less than six for a six-year primary cycle, for 
example) indicates incomplete schools, mult:-grade teaching, or both. 

Table One presents the ratio of teacher per student, teacher j)er class, and teacher per 
school for primary schools in the eleven govemorates of the Yemen Arab Republic 
(Y.A.R.) in 1982/83. It is interesting to note that these teacher utilization data indicate a 
fairly large variation among the regions. The smallest student-teacher ratios are in two 
very rural govcrnorates, Ha'riband Al-Jaw^ while the highest rates are in the relatively 
more urban govemorates of Ibb and Taiz. 1 he 45.7 average for all schools disguises a 
variation from classes in excess of 100:1 in some Grade One classes and very small 
enrollments in some Grade Five and Grade Six classes. 



TABLE ONE 



TEACHER AVAILABILITY MEASURES 
YEMEN ARAB REPUBLIC 
1982/83 





STUDENTS 


TEACHERS 


TEACHERS 


GOVERNORATC 


PER TEACHER 


PER CLASS 


PER SCHOOL 


SANA'A 


43.2 


.65 


3.1 


TAIZ 


48.9 


.87 


5.3 


HODEIDAH 


47.1 


.81 


3.8 


IBB 


.54.6 


.73 


3.3 


DHAMAR 


47.9 


.63 


2.4 


HAJJAH 


38.9 


.59 


2.1 


BEIDAH 


39.6 


.67 


3.2 


SA'ADA 


32.5 


.56 


2.1 


MAHWEET 


43.0 


.49 


2.0 


MA'RIB 


26.8 


.54 


2.1 


AL-JAWF 


26.7 


.53 


1.8 


TOTAL 


45.7 


.70 


3.2 



SOURCE: Ministry of Education, 1984 data reported in lEES Project. Yemen Arab 
Republic Education and Human Resources Sector Assessment, 1986. 



The teacher per class figure varies from .49 to .87 with an average of .70. This illustrates 
that the practice of multi-class teaching is common in most areas. Those govemorates with 
the greatest proportion of small schools (Al-Jawf. Sa'ada, Ma' rib. Mahwcet, and Hajjah) 
have the lowest ratios of teachers per class indie ting theaddi tional incidence of multi-class 
responsibilities in the most rural areas. 
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The number of teachers per school ranges between 1.8 and 5.3 with an average of 3.2. 
In the Y.A.R., the government has been successful in assigning teachers to rural schools; 
one goal of this policy is to have smaller class sizes help offset the educational disadvan- 
tages the rural schools face. 

The final measure of teacher availability is expressed in the number of hours of instruc- 
tional time spent per week in educational activities. In the process section a measurement 
device will be discussed for articulating how teachers actually spend their classroom time. 
The availability measure can be based on official "expectations" or observed behavior. In 
either case it is important to identify teacher functions that take place outside the classroom. 
For example, a report on rural schools in Shanxi Province, China (Study Team on the 
Situation of Rural Schools in Shanxi Province, 1986) noted that middle-school teachers are 
in class only 24 hours per week — about two-thirds of the average for their counterparts in 
Europe and North America, However, in the Chinese system teachers have intense out-of- 
class responsibilities including tutoring slower students, organizing enrichment activities, 
supervision of dormitories, and maintaining contacts with the parents and local commu- 
nity. For the time measure to be meaningful as an indicator of effectiveness one needs to 
know the full range of teacher functions. 

This extended discussion of teacher measures is justified by the centrality of teachers to 
most national systems of instruction. The other input characteristics that follow are 
proposed more as complements to, raiher than substitutes for, the teacher characteristics 
measures. The latter have a crucial role in any consideration of educational inputs as 
indicators of educational effectiveness, 

B. Facilities 

The next category of input indicators to be discussed is facility characteristics. The 
facility characteristics are divisible into issues of size and of availability of special use 
facilities (e,g,, recreation areas, laboratories, and vocational/technical shops). The 
aggregate or average physical size of the school or the classroom is not usually of direct 
importance; these measures may have some effect on construction costs (significant 
economies of scale — savings related to larger size — may exist in initial school construc- 
tion) and there are some potentially negative scale effects in terkns of school administration. 
However, as school inputs and as proxies for quality or potential effectiveness, the most 
important facilities characteristics are the utilization measures of students per school, 
students per classroom, and (where data is available) students per square yard or square 
meter. Inputs as defined here represent availability for utilization rather than actual 
utilization . The measures of students per school, per classroom, or per unit of area are used 
as inputs, not process measures, because they are measures of the availability of space and 
not of the actual form of its utilization. The form of utilization is a process, not an input 
issue, and will be discussed in the section on process indicators. 

Students per school is an intcrpretable indicator only when one knows something about 
the normal physical size of the schools in a country, the nature of the instructional process 
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(i.e., the requirement for special use facilities), and the distribution of population. For 
example, rural schools are almost always smaller, and smaller schools, because of the 
existence of economies of scale, are more costly per student; however, smaller schools are 
not inherently more inefficient. Efficiency is determined in terms of existing constraints 
and available alternatives. If a rural school' s size causes it to cost 20 percent more per-pupil , 
that is inefficient only if some allemative is available to provide the education at a lower 
cost (or if the harsh judgment can be made that the value of educating rural students is not 
worth the extra cost that must be incurred). 

The normal means of resolving the problem of small rural schools is through school 
consolidation. This requires either the commuting of some students to a more distant 
location or the provision of residential quarters for at least some of the students. In most 
developing nations the poor quality of transportation and the isolation of many rural 
communities (accentuated in many locations during the rainy season) often make the 
commuting alternative unfeasible. Transport (either in terms of infrastructure or vehicles) 
may improve over time (as it has in rural areas in more developed nations) and cause the 
alternative of daily student commuting to be more attractive. Provision of dormitories 
often is more expensive than the operation of the individual small schools. In most cuitui-es, 
residential schools are not considered appropriate for children of primary school age. Even 
at the secondary level there may be opposi tion to res iden tial schools that are coeducational ; 
these cultural and social constraints further limit the ability of educational planners and ad- 
ministrators to reduce school costs through consolidation. The important point is for the 
educational analyst to be able to distinguish between a situation of higher student costs that 
represents inefficiency and one that simply reflects tlje inevitabilities of the school location 
and environment.* 

Issues related to classroom utilization indicators are similar to those discussed for school 
utilization indicators. If classrooms are a standard size and there are no distinctions for 
special uses, a very straightforward interpretation can be made. However, classroom size 
may vary by location, by level of education, by type of education, and by the subject matter 
taught in the classroom. Classroom utilization standards will vary by the same character- 
istics. A rural school inevitably may have fewer students per classroom, secondary 
classrooms normally will have fewer students than will primary ones, classrooms at a 
vocational school may be smaller then those found at academic schools, and a social studies 
classroom may be much larger than a science or art classroom. None of these relationships 
are rigid but they do point to the danger of interpreting quality or effectiveness based on 
aggregate or average data on facilities per student 



* It may be noted that urban schools, even with higher utilization rates, can be more costly 
per student because of the higher costs of land, material, or labor in urban areas. This 
would be a more common finding if the land-use cost for urban schools was priced in 
terms of the opportunity value (interest and depreciation) of the land on which the 
schools are builL 
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A more refined measure of the facilities indicator is the use of a unit of area (square meter 
or yard) to denote the average versus appropriate size for a school or classroom. To be of 
value, the measure should relate to the issues of location, level, type, and subject discussed 
above. Only then can an analyst compare the actual availability of facilities with that 
deemed appropriate, minimal, or optimal. 

Some schools may be constructed at a size larger than current use requires. The 
purposeful creation of excess capacity can be justified when changes in population 
distribution or school attendance are expected to increase facilities utilization to the 
acceptable level in the future and where no substantial effect on cost is created in the 
interim. In many areas it may be less expensive (even considering the immediate effect of 
interest paid or foregone on building costs) to create the additional capacity at the time of 
initial construction than it is to make sequential additions to a school as enrollment 
increases. Thus, the analyst must know whether any under-utilization of facilities is a 
transitional or permanent condition of the school or school system. 

Related to the above, aggregate measures of facilities utilization are those measures that 
simply list an inventory of the available special use facilities. It is common within the 
conductof an educational census to coUectdata as to whether a school has a laboratory, an 
administrative office, a workshop, student lavatories, a kitchen, etc. These censuses 
normally cannot provide information about the quality and utilization of these facilities. 
Observation at actual school sites will reveal that significant variation in the nature of 
provision and utilization of 'nccific facilities for instruction or other purposes can exist 
even within a single special-use category. 

The information available on facilities can often be at a level of detail that is surprising. 
For example, in the Yemen Arab Republic, the educational statistics office maintains data 
on the building materials used in school construction (stone, cement, bricks, wood, other), 
perceived overall condition (good, needs repair, poor), availability of special facilities 
(study rooms, administrative offices, bathrooms, storage rooms, and laboratories), availa- 
bility of electricity, and availability of water. The Ministr/ of Education, as part of its 
design of a new educational management information system, is reexamining the facilities 
section of their school questionnaire to be sure the data produc^xi relates to the data needs 
of the Ministry's decisionmakers. 

The interest shown by ministries and other agencies in educational facilities is notewor- 
thy given the low degrceof importance assigned to facilities in most studies of determinacy. 
Fuller (1985), in a survey of World Bank research, confirmed earlier analyses that have 
found that while some correlations exist between facilities quality or special-use facilities 
availability and student achievement, these correlations are consistently small and of 
questionable policy significance. Windham (1986) noted: 

While some minimum facility quality undoubtedly is required in most environ- 
ments, and there is a persuasive case to be made for facilities quality as a constraint 
on school learning, there is no similar case to be made, intuitively or statistically. 
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for facilities construction as a major vehicle for efficiency enhancement. The 
status of facilities utilization is a more critical issue than the simple availability of 
schools built to Western standards. 

C. Equipment 

The discussion of c^iiupment inputs as indicators of educational quality or effectiveness 
parallels that just presented for facilities inputs. Utilization measures will always be 
superior to availability measures as indicators of educational effects. However, availability 
measures may be all with which one has to work. The most important types of equipment 
one normally considers are laboratory equipment for the physical and natural sciences; 
vocational/technical equipment used in woodworking, metal working, electronics, practi- 
cal engineering, and related subjects; and audio-visual equipment used in support of 
instruction (the traditional forms such as radios, film projectors, tape recorders, and 
overhead projectors have now been supplemented by the newer technologies of television, 
vidco-casscttes, computers, and compact discs; Block. 1985). 

The availability of even the most basic equipment is still a rarity in the primary schools 
of most developing nations and the incidence of all equipment is biased toward urban areas 
and the more developed and politically powerful regions of a nation. The reasons for this 
are simple. The schools that arc advantaged in terms of their location in urban areas arc the 
easiest to which to disseminate new equipment These schools are also the most likely to 
have the electrical supply needed to operate much of the equipment and the teachers who 
arc best prepared to use the equipment effectively. The result of this coincidence of 
locational advantage and access to educational equipment is that equipment availability is 
a very good correlational indicator of school quality and potential effectiveness whether or 
not a causal relationsliip can be established. The availability measure indicates the direct 
effect of additional insuuclional resources available to students as well as serving as a 
proxy for the complex .nix of favorable economic and social biases indicated by the concept 
of locational advantage. 

The use of the equipment measure as an indicator at the educational system level is more 
complex. Here, the proxy for locational advantage is diffused because one is working with 
aggregate or average measures of availabil ily; increases in these latter measures should still 
be useful as indicators of systemic increases in quality and potential effectiveness because 
the measures do reflect an increase in the availability of insuiictional resources in the 
system. The measurement of the incidence of availability can be used to create valuable 
indicators of system equity. To the extent that availability is biased toward the already 
advantaged locations, the provision of equipment may be seen as reinforcing inequity by 
conu-ibuling to the convergence of disadvantage faced by students in remote and rural areas 
and the least developed regions. As educational equipment becomes more equally 
distributed among schools, its measurement will be a less useful indicator of quality. For 
a considerable time, however, this input, even in the absence of utilization data, will 
continue to play a potentially important role as a marker for the identification of advantaged 
and disadvantaged schools. 
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D. Educational Materials 

The availability of educational materials has received increasing attention in the last 
decade because of growing evidence that it is an important correlate and a probable 
determination of classroom achievement (Heyneman, et. al. 1978 and 1983; Searle, 1985; 
Fuller, 1985; Suryadi, Windham, and Green, 1986; and Lockheed, et. al., 1987). The 
attention directed to educational materials has been divided between the question of 
providing basic textbooks (Heyneman, Farrell, and Sepulveda-Stuardo, 1978) and the 
introduction of modular programmed learning materials into classroom use (Cummings, 

1 986) . In both cases the materials are seen as having some effects that complement existing 
teacher skills and other effects that substitute for teacher inadequacies (Lockheed et al 

1987) . * *' 

In terms of aggregate availability and the incidence of availability, textbooks are a less 
extreme case than is educational equipment. Increasingly, in most developing nations 
outside sub-Saharan Africa, textbook distribution efforts are increasingly successful. The 
financial support of international donors and the efforts of indigenous curriculum officials 
also have resulted in an improvement in the overall quality and local relevance of textbook 
materials in the last decade. Where these efforts have been successful, the availability of 
textbooks is no longer a useful indicator, by itself, of school quality. Also, in these nations, 
textbook availability is less of a force to magnify existing inequality tlian was the case when 
the textbooks were available only or primarily in the socially advantaged locations. 

In much of sub-Saharan Africa, and in some poorer countries elsewhere, the conditions 
of textbook availability are still at a critical stage. In some cases, such as Liberia and 
Somalia, potentially valuable textbook design and adaptation efforts are frustrated by the 
problem of distribution. The distribution constraints relate primarily to problems of 
finance, transport infrastructure,and the administrative capacity to manage the distribution 
effort. In other countries such as Cameroon, Kenya, and Botswana, the focus is on 
improving textbook utilization through provision of training in the pre-service and in- 
service teacher preparation programs. Throughout the developing world, better plans and 
policies are needed to deal with the three stages of instructional materials dissemination: 
development, delivery, and utilization (Windham, 1986). 

In nations that have a large preponderance of underqualified and unqualified teachers, 
textbooks have a special role often underappreciated by ministry and donor officials. In 
addition to the traditional functions as an information resource and a curriculum design 
format, instructional materials can be a training device for the less qualified teachers. 
Teachers, by following the sequence and content of textbooks, programmed materials, or 
by using otlier instructional support supplies (maps, charts, diagrams, special-topic 
booklets), acquire both new knowledge and an appreciation for the principles upon which 
classroom organization for instruction are based. In the better textbooks and materials the 
design principals are more explicit as each learning unit includes new information, 
examples, questions, and even sample examinations. Even in the less well-designed 
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materials a system of organization and a rationale are implicit and, over time, improved 
methods can be acquired by the conscientious teachers. 

Programmed instructional materials, whether designed for use in activities led by the 
teacher or more independently by individual students or student groups, offer the most 
explicit instructional design formats. However, whether explicit or implicit, instructional 
design characteristics of classroom materials will continue to be a major determinant of 
their value for on-the-job learning by teachers. Thus, the availability of these materials has 
implications not just for immediate quality and effectiveness but also for the long term 
developmental effectiveness of both teachers and materials. 

Two major cost issues exist concerning instructional materials: the determinants of 
production and distribution costs and the responsibility for textbook financing. The 
determinants of textbook costs include a myriad of factors related to availability of paper, 
local printing capacity, the nature of the existing transportation infrastructure, and 
administrative capacity for the management of development and distribution activities. A 
major policy issue for many developing nations is the decision for internal or external 
publication of instructional materials. In nations such as Indonesia or the People's 
Republic of China, the economies of scale are enormous and thus internal publication is 
easy to justify. For smaller nations, and especially those smaller nations with a unique 
language such as Somalia, the policy problem is much more difficult to resolve. Internal 
publishing resources may not be adequate to meet the demand for educational materials in 
the local language and yet the relatively small quantity and specialization required for 
external publication of these materials results in the smaller nations often incurring a much 
higher unit-cost for instructional materials than would a larger nation. 

Internally, distribution costs may be a significant retardant to efforts to disseminate 
materials equitably to remote rural areas. This issue of cost has an important convergent 
effect with the question of textbook financing (Mingat and Psacharopoulos, 1985; Jimenez, 
1986; World Bank, 1986). In those nations where textbook costs are solely a responsibility 
of the government, the rural poor are left free of the burden ofpaying for textbooks but, too 
often, also <u-e the last ever to obtain textbooks if they receive them at all. In contrast, where 
textbook costs arc charged fully to the student or parent, the real sacrifice required to 
purchase textbooks may be greatest for those individuals in remote areas who have low 
incomes and also must pay higher prices because of the extra distribution cost to deliver 
books to rural or remote areas. A compromise system used in China and some other nations 
is that the government textbook monopoly provides textbooks everywhere at the same 
price; thus, the government bears the cost of distribution expenses outside the urban areas. 
Also, government at the various levels can decide to subsidize textbooks for certain 
geographical areas or for disadvantaged families. Partial subsidization in this form has the 
disadvantage of requiring a needs-basis or other criteria for judging who receives the 



47 



37 



Chapter 3 



subsidy; however, general subsidization also has a cost in terms of government payment of 
charges that could be borne by more advantaged members of society.* 

Instructional materials availability and costs are commonly used indicators of quality 
and effectiveness. A final issue in regard to these indicators is the assumed proportions 
necessary between instructional materials and students. Many textbook distribution 
schemes assume it is necessary to have one textbook (in each academic subject) for each 
student. While this is the common current pattern in most Western schools, it has not 
always been so and does not represent a functional requirement for instruction as much as 
a convenience for classroom management and a facilitating device for user-financing. 
More research needs to be done on alternatives of multi-student use of textbooks and the 
possible negative effects this may have on learning and on the teacher time required for 
classroom organization activities. At present, however, the ratios of textbooks or other 
instructional materials per-student, except at the extreme values, may be better indicators 
of family income or governmental fiscal capacity for education than they are of potential 
instructional effectiveness. 

A final point that relates cost to effectiveness issues is the policy decision to change the 
approved textbooks or other materials. If the officially approved textbooks are changed, 
for example, then the result is to devalue all existing materials in the schools, Tnis can be 
an especially damaging decision where families have "invested" in textbooks t;nder the ex- 
pectation that the books will be available for reuse by the families' younge: children or 
could be resold in the secondhand textbook market that often exists in even the smallest 
villages. Regardless of whether the books are owned by families or by the school, the 
decision to replace textbooks can have the effect of wasting an educational input The 
decision must be based on confidence that the advantages of the new textbooks, in terms 
of additional effectiveness, will offset the transitional increased marginal cost of abandon- 
ing the previous textbooks. The negative effects of a policy decision to introduce new 
educational materials can be minimized if a transition period is allowed. Given that 
textbooks rarely have a usable life of more than three or four years in developing nations, 
the concept of a transition period is not a difficult one to implement. 

Subsumed within the instructional materials input category- is the issue of curriculum. In 
the past, curriculum quality issues (including national or local relevance) have been 
ignored in most studies of educational effectiveness or efficiency. It is not that economists 
are unappreciative of the potential role played by curriculum in determining both costs and 
effectiveness; the problem is that no agreement appears to exist, even among curriculum 



* Thobani (1984) suggests that the common convergence of inadequate government fiscal 
resources and excess demand for certain educational services justifies consideration of 
the wider use of "user-fees" for education. Klecs (1984) objects to what he sees as 
Thobani's limited analysis and suggests a wider "political economy" view. Both authors 
stress equity outcomes as a critical consideration in user-fee effects on educational 
efficiency. 
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specialists, as to how to measure and value variation in curricula.* Until some consensus 
of opinion evolves or is mandated, curriculum inputs will continue to be underemphasized 
and will manifest themselves in studies of effectiveness and efficiency in terms of process 
indicators rather than as input indicators. The sole exception to this will be where the 
curriculum defines the quantity and nature of instructional materials to be used. In that case 
the instructional material inputs discussed above are joint indicators of materials and 
curriculum. 

E. Administrative Capacity 

The lack of attention paid in education production studies and analyses of effectiveness 
and efficiency to the influence of administration is puzzling. The literature suggests an 
increasing confidence that administrative capacity is a key variable, especially in determin- 
ing performance of the best and worst schools (Paul, 1983; lEES 1984A,B,C, 1986A,B; 
Glasman, 1984). However, as was the case with the curriculum variable, there is little 
consensus as to how best to operationalize the concept of administrative capacity. 

The most common measure used to indicate administrative competence is the educa- 
tional attainment of the administrator. Sometimes this measure is refined to reflect specific 
exposure of the administrator to training in management and planning skills. Also, as with 
teacher input proxies, experience measures have been added in some specifications; this is 
operationalized as age, years of employment, or years of administrative responsibility. 
Conceptually, the experience factor should be at least as important as that of formal or 
specialized training since on-the-job learning is such a critical determinant of administrator 
competency. This is even more the case in developing nations where administrators are less 
likely to have substantial formal training in management science or decisionmaking. 

In addition to the personal ability of the administrator, educational administrative 
capacity includes the appropriateness of organizational structures, individual and group 
attitudes toward hierarchical systems, the range of available personnel incentives, and data 
availability and utilization. Personal experience suggests that the ability of administrators 
in many educaUonal systems is less a problem than the inefficient bureaucratic structures 
within which they work, the social and cultural constraints on hierarchical decisionmaking, 
the limited incentives or sanctions that the administrators can apply to subordinates, and the 
lack of appropriateness, quality, and timeliness of data upon which the administrators must 
base decisions. Improved pre-service and in-service training for present or future admin- 
istrators can be a positive influence (and thus of some value as a statistical measure) but the 
improvement of educational efficiency will require an integrated approach to all of the 
factors that ultimately determine administrative capacity. 

This is a special case of a general point. Some measures may exist that are adaptable to 
quantitative and statistical analysis of effectiveness and efficiency. However, these are 

* However, an interesting example of the capacity of outcome-based instruction to 
enhance learning is presented in Jones and Spady (1985). 
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never the only important measures and. in cases like that of administrative capacity, the 
quanutaUve or statistical measures may not even be the most important 

The following section on process variables will include some addiUonal indicators of 
admmistraUve behavior that probably are better proxies of the capacity of the school or 
system administrator than the input measures of fonnal or specialized training and 
experience. However, the input measures mentioned here are of some value and have the 
major redeeming quality of being readily available as part of most educational statistical 
systems. 

F. Summary of Input Measures 

This section on inputs has emphasized those measures that represent the availability of 
resources to the classroom, school, and system. A measure of potential versus actual use 
IS inherenUy inferior in measuring effectiveness although it may be an acceptable proxy 
measure of costs. With certain excepUons resource costs in education are determined by 
whether an item is made available not whether it is used. The important exception to this 
are certain materials or equipment which will not deteriorate as rapidly if not used. 
However, given maintenance conditions in the majority of educational institutions, even 
equipment will not be fully preserved simply by postponing its use. Also, the financi^ cost 
of supply IS immediate and the postponement of use may only reduce the present value of 
the beneficial effects that eventually are realized (by shifting the incidence of benefits to 
a more distant time). 

While the input measures are inferior to process measures they can prove to be cost- 
effccuve as data. The next section will indicate that the superior insight gained from the 
use of process variables often is purchased at a higher price in terms of time and labor costs. 
If one adds to this the additional limitation that process measures collected through 
observational methods involve an inherent loss of generalizability, the conceptual advan- 
tages of process variables may not always justify their additional cost. 

The point is that the modesty, and at times skepticism, directed toward the use of input 
measures is not always justified. Input measures are more readily available and often 
controlled more direcUy by the educaUonal authority. While one may prefer to affect 
classroom or school behaviors and attitudes direcUy, one normally can only change those 
input characteristics of the participants that one believes are correlated with the desired 
behaviors and attitudes. Thus, effectiveness or efficiency enhancement acUvities must be 
understood as part of a process wherein the planner or administrator attempts to maximize 
the probability of increased effectiveness or efficiency based on: (1) t^/e available 
information on inputs and their determinacy for process effects; (2) the probable relation- 
ship of process variables to desired outputs and outcomes; and (3) the probable costs of 
reforms relative to the expected availability of resources. In this situation certainty is 
impossible and a normal bureaucratic response to uncertainty is inaction. 
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The educational system's senior officials can combat this adminisuative inertia only by 
creating a management system that encourages responsible experimentation. As more is 
learned about process phenomena. educaUonal outputs, and social outcomes, the results of 
basing educational decisions on available input measures will be improved. The skepti- 
cism toward the use of input measures in determining educational policy and practice is 
justified only when such measures are used in isolation from the more immediate processes 
and effects of education. 



XL PROCESS INDICATORS 

The analysis of educational process is a study of the interaction that takes place among 
inputs under different forms of classroom technologies (instructional systems). Because 
interaction among inputs rather than the acUon of individual inputs is the focus, the 
discussion of process indicators cannot follow the same outline used for discussing inputs. 
Rather, the discussion presented here will emphasize three aspects of analyzing the 
educational process: (1) the analysis of administraUve behavior; (2) the analysis of teacher 
behavior (with an emphasis on patterns of time allocation); and (3) the study of specific 
student behaviors related to time on task and observable uUlization of school-provided 
resources. 

It should be made clear immediately that process variables are the least suited to survey 
analysis and. to be measured properly, normally require observational data collection. 
Thus, the analyst must be prepared to jusUfy the decision to study process issues rather than 
simply the common variation of input and output indicators measured at a higher level of 
aggregation. Theisen. et. al. (1983) note that (p. 67-68): 

...national studies are beset by a host of linguistic, logistical, and methodological 
problems....The time is ripe to move from aggregate, descriptive studies of 
determinanls...to those that will be of use in vitalizing efforts to improve 
educational outcomes. 

In the following discussion, an attempt is made to indicate the advantages of studying 
process variables. 

A. Administrative Behavior 

Given the availability of adminisuative input data in the form of educational attainment 
levels and years of experience, the first type of process data that might be collected relates 
to incidence and form of administrative monitoring. One example would be the frequency, 
length, and purpose of visits by school inspectors or advisors. It is recognized that the role 
of school inspectors varies greatly within and among developing nations and that the 
appropriateness of inspector training to their level of responsibilities is a point of 
controversy. However, if the purpose of visits (whether to police or to advise, whether 
aimed to monitor school administration or classroom instruction) is known, as well as the 
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frequency and lengdi of visit, then it is easier to interpret this data in terms of normal 
production relationships. 

Without the knowledge of purpose, one can generate some anomalous statistical rela- 
tionships. For example, if visits by school inspectors primarily are in response to admin- 
istraUve or instructional problems, one will find a negative correlation between the admin- 
istrative indicator and school performance in the short run. This is a situation analogous 
to what one finds in the field of health; if medical personnel visit only when disease or injury 
exists then the fact of their visit is an indicator of a problem. However, if medical personnel 
visit primarily to promote improved health and safety, one would expect, over time, a 
positive correlation between frequency and length of visits and the resultant quality of 
health. ^ ^ 

Similarly for the school, if visits of supervisors primarily are related to improving school 
administration and classroom instruction, then, over time, a positive relationship with 
school achievement measures should occur. If a positive relationship does not occur, one 
is forced to question the value of the inspectorate program. Either the supportive functions 
of the inspectors are not properly designed or the inspectors themselves are not adequately 
trained or motivated. 

The example of the inspectorate is an excellent one to indicate the relative value of 
process versus input indicators. If one used only input measures for the valuation of the 
inspectorate role (e.g.. number of inspectors, level of training, length of experience, 
inspectors per school or per class, inspector salaries or total inspectorate costs), one could 
be seriously misled about the actual role of the inspectorate. A common phenomenon in 
the poorest nations is that whi le an inspectorate exists it is constrained from fulfilling its re- 
sponsibilities by the shortage of funds for transport and per-diem costs. Thus, some 
inspectors do not inspect any schools and most inspectors find it exceedingly difficult to 
visit the more isolated schools (the very locations most in need of external administrative 
and instructional support). Even in the more advantaged nations, transport limitations can 
act as a serious constraint on transforming this potential resource into an input that directly 
affects school performance. 

A second set of administrator process indicators would be those that measure the school 
administrator's interaction with teachers and pupils. Again, data on frequency and length 
of interaction will be useless without knowledge of purpose. Just as with external visits 
from inspectors, the internal visits by school administrators can be either to respond to 
existing problems or to prevent future problems. All school administrators undoubtedly 
will have some interaction of the first kind; the data question is the relative incidence of 
visits that involve "pc>licing" teacher and pupil behavior versus those that involve support 
of classroom management, instruction, and individual and group learning. 

The final major area of administrative behavior, and one that commonly is ignored by 
both survey and observational research, is the interaction with parents and c^-mmunities. 
Contacts with parents have three important aspects: to encourage parental support of 
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educaUonal activities of the family's children; to promote parental and community 
^voWement in the education process itself; and to interact with the community concerning 
^^^^Wemsof schooldiscipline and poor student perf^^^ 

rSdministratormLtingswithindividualparentsandparentgroupsd^^^^^^^^ 

™e of education is explained as are the school's expectauons of the students. While 
Sraprollitizingactivity.thisisalegitimateadministradvefunctionand^^m 

whirl m^y p^ents do not have educational experience themselves, a crucial one. 

Thepromotion of parental andcommunityinvolvementhasthreedcsiredoutcomes:(l) 

utiStLThome Loui.es in the education process; <^^y^::^^;'^^ZlZ^l 
members in instructional and instructional support roles; and (3) parucipauon of parents 
rdcrmmunity inprovidingfinancialsupportfor the school. Theabihty o^ 
Tte sup^rule of schoof instruction obviously is limited by the educational level of 
oareni eCSly that of the mother who is likely to play the central role m assisting 
SXschilworkathomeandinaffectingthei^^ 

.rhool Dolicie^such as use of a language of instrucuon different from that of the parents 
or oT cu^cul^^^^ such as modern mathematics (emphasizing n-ber 

o m«lem science (such as the Nuffield science P-f-,>^- ^ ^"^^f^^ ^^^^^ 
of parents or community members to assist student learning. J" /^oi^f f ' 

advan^gc of concurrent adult education in literacy and numeracy is that it has the ability 
to promote adult support and empathy for the learning processes of children. 

The use of community resources in direct instruction in the classroom s"mlarly is 
cons^ained by the educaL level and skills of the community members; unfortunately 
suruse also is constrained by conservative attitudes on the part of admmistralors and 
ethers wh^ ^e reluctam to Encourage "external" involvemem in their acUvme. Th^^ 
communnY participation can be especially meaningful in providing craft ^^f^^^^^^ 
b^yoTthe areas of competency possessed by the teachers and m assisung teachers or 
substituting for them during periods of absence. 

Finally.theadministrator'spaiposeinencouragingparentalandcommuniiW^^ 
be designed to provide enrichment funds as a supplement to govemmen funding or. in 
mivate eSucaUon can be required to assure the very existence of the school. The paymen 
of ;SiSol fees for iSoratory expenses and of textbook charges may essen^hf 
a 1 children are to have access to more equal educational resources in the classroom^ 
N^^o W dng pints to pay these amounts can be difficult in economically disadvantaged 
«unS but the failure to do so can Lad to ^Zl 
advantaged communities and can accent the internal variation within the school between 
those who do buy these materials and those who do not 

The effective school administrator must deal with parents or community leaders con- 
cembg Sems of school discipline or poor studem performance. The enforcemem of 
scTooTrul^^^^^^^^^ 

Tnd li sup^rtSnship can be affected by the social and political roles the schoo oc- 
clnal yTforct^^ assume (Salifu. 1985). The need to deal with complaints from 
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behavior. Wh mlTr,^ f of adm.msMors rate ton U,e success of Ihe 

collccB^ldirecT S ripSLfr 'r ■"""'■^ Panicipadon should be 
be done in 0,e u^de^uSZ s.^,? 

generated ralhe, 1 ' T"""?' " ""*emen, may be sclf- 

crucia, corapiemZ^^Tr^rr^orTrschir r ' 

..ndin, Of i aL^s^rjn'r pirratn?^^^^^^ 
B. Teacher Time Allocations 

ca:^s«2r<tdt^^^^^^^^^^^ 
.=srporsibr"^''^'^--=~- 



44 



5- 



Indicators of Effectiveness: Inputs and Processes 



The Umc allocauon pattern of the teacher's instructional acuvity may be depicted as ^ 
three by four matrix as indicated in Table Two. * The horizontal categones relate to the 

fn this case it is obvious that the teacher depends on full class lectures for the 
Sri y of dti n"m:cuon (with a comparable distribuuon of preparauon u es 

TnSu^ student contact a. the major form of remediauon. f -^^^^^l^^^-^^f^ 
(20% versus 60%)asmuchtimeasdirectinstrucuonandisagamdommatedbythefallclass 

lecture approach. 

TABLE TWO 

HYPOTHETICAL MATRIX OF TEACHER TIME ALLOCATION 
Sp'^ PRI-PARAllON INSTRUCTION REVIEW REMEDIATION TOTAL 

FULL CLASS 6% 
SUB-GROUPS 2% 
INDIVIDUAL 2% 



TOTAL 10% 



40% 


15% 


4% 


65% 


10% 


2% 


1% 


15% 


10% 


3% 


5% 


20% 


60% 


20% 


10% 


100% 



U must be remembered that the percentages used in the table refer only to ume spen t on 
the nTucUonaTrnk If the teacher normally spends 15% of the school day on^n.^^^^^' 
tLn and foTon m^^^ and evaluation, then the remaining 75% is available for Oie 
nTtr^aio^^^^ 

time spent on a task one can multiply the percentages m the table by the 75% ot ume me 
lie Cesents of all activities. For example, full class remediauon takes 4% of 
instructional time and 3% of all time (75% times 4%). 

Th. rtivi.ion of categories in a time allocauon mauix are not fixed but should follow 

classroom environment. 
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A^aluablepuiposeoftheUmeuseindicatorsisnottoidenUfyeffectiveness ner^. h„, 

proacnes, it is possible to cone ude that a new annrnarh hoc r^n^A . - ^ 
performance when, in fact, closer study SSvioTl^r" X^^^^^^ 

doprov,deabasisfor.akingsounderinferencesaboutthT^^^^^ 

the nature of resource interaction within the classroom. '^acner resource and 

C. Student Time Allocations 

Process measurement of student behavior follows much the same nattem a. for ,...h.. 
behavior. Surveyapproaches.depcndingoneitherteacherorstude^^^^^^^^^ 
allocations, are acceptable but generally considered inferior to the m"ement hv 
Observational techniques. The measurement of individual studenrbc^vroH ^bTiuo a 



* 'r'^*s*f^"exampleofwhatDobsonandCooic(!9S0)refertoasTypeIIIevaluatione, 
namely, the evaluaUon of a program or treatment that, in realitrhas n^ c;S:;<S 



error; 
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teacher present, (4) individual tutorial with teacher, and (5) working alone. The second 
dimension of behavior is the form of materials used; here the categories are ( 1 ) no materials, 
(2) textbooks, (3) instructional support materials, and (4) audio- visua! equipment 

In tlie example given, fully one-half of the student's time is speut in listening to lecture/ 
discussion presentations without the use of any instructional materials. The next two 
largest categories of time are textbook use in a full class setting and textbook use alone by 
the student. In this example, support materials are relatively heavily used while audio/ 
visual equipment is rarely used and only in the full class setting. 

Again, time allocation data on students are not direct indicators of effectiveness or 
efficiency but do provide more informed judgments to be made about whether the 
instructional process is using resources properly and what the probable effects of instruc- 
tion will be. This data, unlike most other measurements, can lend itself to the discussion 
of possible attitudinal and behavioral effects of instruction. Because each form of 
interaction elicits different patterns of student behavior, it is possible to suggest different 
probabilities for such traits as independence, leadership, or cooperation based on student 
time allocation data (this is especially true when the data is generated using observational 
techniques). 



TABLE THREE 
HYPOTHETICAL MATRIX OF STUDENT TIME ALLOCATION 



FORM OF 
MATERIALS USE/ 
STUDENT INTERACTION 


NO MA- 
TERIALS 


TEXT- 
BOOKS 


SUPPORT 
MATERIALS 


A/V 
EQUIP. 


TOTAL 


FULL CLASS 


50% 


10% 


8% 


2% 


70% 


SMALL GROUP 
WITH TEACHER 


2% 


2% 


1% 


0% 


5% 


SMALL GROUP 
WITHOUT TEACHER 


1% 


6% 


3% 


0% 


10% 


TUTORIAL 


0% 


0% 


0% 


0% 


0% 


ALONE 


0% 


10% 


5% 


0% 


15% 



TOTAL 53% 28% 17% 2% 100% 
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In designing the measurement of student behavior one can add new dimensions (empha- 
sizing facility use, active ve sus passive student behavior, or var>'ing subject matter 
emphasis, for example) or define new categories for the dimensions given (one may have 
a category for mixed-materials jse since, for example, a textbook could be used in 
conjunction with other materials or with audio-visual equipment). These decisions are of 
less immediate concern than the recognition by analysts that the time- use and other 
behavioral data on students are a legitimate means of assessing quality and probable 
cffu:tiveness or efficiency. 

The data presented in Table Three represent only the time on-task. Any time allocation 
study wouM also have a separate classification for timeoff-task (either as a single category 
or as a behavioral dimension with a set of separate sub-categories of its own). However, 
the off-task time normally would not be included in the time allocation matrix since it 
would not cross- tabulate with the other dimensions (such as materials use). If 25% of the 
student's time is off-task, then the percentages presented in Table Three are percentages of 
the 75% of time on-task. For example, the 8% of time spent in full class use of support 
materials would equal 6% of the toul student time (75% of 8%). 

There are special applications of the student — and the teacher — time use data to analysis 
of equity effects. By analyzing student interaction with resources by student category 
(gender, ethnicity, social class) within a single classroom or school, one can develop a 
much more reliable indicator of probable inequality in achievement, attitudes, and 
behavior. It is interesting to observe whether students with existing learning disadvantages 
receive more or less attention from teachers and whether they are subject to a difference in 
the form and amount of tutorial or small group assistance. Educational input data that 
exclude these issues of process may indicate that disadvantaged students have potential 
access to equivalent teachers and other resources. Process measures of the same students, 
however, could indicate that the disadvantaged students receive substantially less direct 
access to teacher time, material resources, and peer support. Thus, different measures of 
the same classroom can result in indicators of effectiveness and efficiency that are 
intcrpretable in diametrically opposite ways. 

Student behavior data, of the type discussed here, can be of value when used independ- 
ently but is of greatest use when combined with other process data on administrators and 
teachers. Collectively, the behavioral data can give a more complete explanation of how 
all of the resources of the classroom and school interact to produce educational effects. The 
relevance of this to educational planning and management is three-fold. 

First, to the extent that adminisu^ator and teacher behaviors can be correlated with 
administrator and teacher characteristics, there will be a greater ability to interpret the 
effectiveness of education in those cases where only input data are available. The 
information linking characteristics to behaviors can serve as a basis for teacher/administra- 
tor selection criteria as well as helping to design teacher/administrator training programs. 
Second, the knowledge of effective behaviors can allow the curriculum for training 
programs to be further refined so as to develop the desired behavioral patterns and to allow 
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selccvion of training candidates on the basis of conducive attitudes and behaviors. Third, 
this same information would give school administrators a better foundation for monitoring 
and evaluation of teachers and a basis for encouraging modification in observed teacher 
behaviors. 

At present, these contributions to the management and planning of education are unre- 
alized. Until a strong research commitment to process analysis is made the contribution 
never will be realized. One must overcome concerns as to subjectivity, immediacy, and 
generalizeability and convince the educational system's leaders that ignorance of educa- 
tional process is a major source of systemic inefficiency and a major barrier to effective 
educational reform. 

Such work is an ideal activity for donor support since it is experimental but does allow 
for the production of generalizable techniques (and possibly generalizable findings). Such 
work requires a commiunent of substantial time as well as resources. However, it would 
engender a valuable discussion of what is wanted from the educational process and what 
can and cannot be modified in the classroom environment. Because of collection costs, 
process data may never be as cost-effective in facilitating educational decisionmaking as 
are input data; however, they can be more cost-effective in promoting correct decision- 
making. 



CHAPTER FOUR 



INDICATORS OF EFFECTIVENESS IN EDUCATIONAL PRODUCTION: 
OUTPUTS AND OUTCOMES 



In this chapter the discussion of educational effectiveness indicators will proceed to Ifc 
two most commonly used categories of measures, educational outputs and educational out- 
comes. As with the earlier discussion of inputs and process indicators, the focus of the 
discussion will be on the value of the indicators in increasing the understanding of 
educational production relationships and of assessing educational effectiveness. 

L OUTPUT INDICATORS 

To many persons, the use of input or process data to measure educational effectiveness 
is anti-intuitive. To them, effectiveness can only be indicated by what the school produces. 
In this section, educational outputs (the immediate effects of the educational activity) will 
be reviewed in four categories: (I) attainment effects, (2) achievement effects; (3) 
attitudinal/behavioral effects; and (4) equity effects. While this categorization does not 
exhaust all possible educational outputs it does encompass the large majority of those 
measures that commonly are used as indicators cf classroom, school, or system effective- 
ness. 

A. Attainment Effects 

The simplest measures of attainment effects are those provided by educational enroll- 
ment statistics. From these statistics one can compare over time the number of students by 
grade or level of education, by program type (e.g., academic versus vocational, secular 
versus religious), by control (private versus government), and by subject specializations 
(these normally are used only in secondary and post-secondary institutions). Tnese 
statistics may be used for comparisons over time at the system, school, and classroom level 
or for comparison among schools and classrooms either within or among the program, 
control, and specialization types. 

Normally, increased attainment is considered a positive indicator of effectiveness since 
a desired output of education is more graduates. Educational attrition and repetition, on the 
other hand, reduce or slow attainment ?nd, therefore, are considered negative indicators. 
It is useful, however, to note that high attainment rates can be achieved by lowering 
attainment standards. Conversely, high standards forattainmentcan result in higher levels 
of attrition or repetition. These points are made to indicate that attainment data, without 
complementary data an achievement, are inherently inadequate measures of educational 
effectiveness. (Sec Haddad, 1979, for a discussion of the educational and economic 
implications of promotion and retention policies). 
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Rates of educational progression, repetition, and attrition can be calculated either from 
a cross-sectional or cohort format Table Four presents a set of cohort data for the Yemen 
Arab Republic (YAR)for 1976/77 to 1982/83. The data on number of schools andnumber 
of classrooms indicate how rapidly the YAR*s educational system exp^ded over that time 
period. The cross-sectional (single year) data for 1982/83 could be used to indicate the 
relative size of different grade levels as a percent of the previous grade, as given below: 



Primary School Enrollments: 1982/83 



Grade Two as percent of Grade One = 81.12% 

Grade Three as percent of Grade Two = 79.91% 

Grade Four as percent of Grade Three = 68.54% 

Grade Five as percent of Grade Four = 63.98% 

Grade Six as percent of Grade Five = 68.22% 



One also can attempt to approximate growth of the Grade One class by calculating Grade 
One enrollment in 1982/83 as a percentofGrade One enrollment in 1981/82. However.in 
the YAR this percentage (97.49%) is less that 100% because of the atypical decline in 
Grade One enrollments between the two years. In the two previous years the ratio was sub- 
stantially in excess of 100%; 113.91% in I981/82and 114.37% in 1980/81. This anomaly 
is an indication of the problem of relying on cross-sectional data, even when comparative 
cross-sectional data is available for more than one year. 

The decline in Grade One enrollments in 1982/83 is explained by the phenomenon of 
multiple age groups of students entering Grade One when new schools first open. When a 
village that previously has not had a school first receives one. students older than the normal 
Grade One student of age six or seven may enter Grade One. Thus, in subsequent years the 
total enrollments in Grade One may fall even though the number of Grade One six- or 
seven-year olds actually may increase. 

An alternative to the cross-sectional student progression data presented earlier is 
possible if one calculates progression as a percent of the previous grade in the previous 
year. When one has successive year cross-sectional data of the type in Table Four this is 
possible. The result of such calculations are indicated as follows: 



Grade Two (1982/83) as % of Grade One (1981/82) = 79.08% 

Grade Three (1982/83) as % of Grade Two (1981/82) = 88.90% 

Grade Four (1982/83) as % of Grade Three (1981/82)= 90.01% 

Grade Five (1982/83) as % of Grade Four (1981/82)= 86.52% 

Grade Six (1982/83) as % of Grade Five (1981/82)= 91.63% 
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TABLE FOUR 

YEMEN ARAB REPUBLIC PRIMARY SCHOOL ENROLLMENTS BY GRADE 

GRADE LEVEL 

TOTAL 



YEAR 


ONE 


TWO 


THREE 


FOUR 


nvE 


SIX ] 


ENROLL- 
MENTS 


1976/77 


86,463 


47,971 


35,292 


23,426 


15,235 


11,772 


220,159 


1977/78 


91,804 


57,784 


41,729 


28,081 


18,184 


13,704 


251,286 


1978/79 


97,288 


58,847 


40,837 


25,596 


16,014 


13,385 


251,967 


1979/80 


140,215 


70,491 


49,640 


33,279 


25,138 


16,486 


335,249 


1980/81 


160,361 


96,381 


65,232 


43,796 


27,640 


20,863 


414,273 


1981/82 


182,666 


129,845 


87,887 


58,499 


37,682 


26,417 


522,996 


1982/83 


178,075 


144,455 


115,428 


79,112 


50,613 


34,529 


609,212 



SOURCE: lEES Proja^*, Yemen Arab Republic Education and Human Resources 
Sector Assessment, 1986. 

Normally, in an (expanding educational system, the progression rates will be higher for 
a comparative cross-section than for a one-year cross- section. This is because the previous 
year's enrollments at each lower grade level will be smaller than the current year's. Again, 
the Grade One to Grade Two progression rate is an exception in the YAR example because 
of the anomaly of the 1982/83 Grade One enrollment decline. 

Even this refined calculation of progression excludes an important factor. While it is 
obvious that most of this ycar*s enrollment in a given grade should have originated in the 
previous yeax's prior grade level, the existence and effect of grade repetition cannot be 
determined from cross- sectional data. For example, in the YAR data, a probable 
explanation of the higher progression rate at the Grade Six level is not just that more Grade 
Five students progress to Grade Six but that Grade Six students are more likely to repeat 
than are students at other grades. Thus, the progression rate, as it normally is calculated 
based on aggregate data, compounds progression effects with repetition effects. Repetition 
levels tend to be highest in the earliest grades (where the requirement to learn a new lan- 
guage or basic skills may hold students back), where national tests are administered, or 
where purposeful bottlenecks appear in the system. The last two locations are often the 
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same and frequently coincide with the administrative division of schooling — that is, 
between primary and secondary, between junior secondary and senior secondary, and 
between secondary and higher education, for example. 

It is important in studies of progression rales to distinguish whether the rate is calculated 
based upon graduation (leaving one grade level) or further attainment (entering the next 
grade level). In the Y AR, for example, the calculation of primary school progression rates 
can use graduation from Grade Six or successful access to Grade Seven as the final standard 
of progression. For 1982/83, the numb^ of Grade Seven students was equal to 77.0 percent 
of the Grade Six graduates the prior year. Thus, if one had calculated overall progression 
from Grade One in 1976/77 to Grade Six in 1981/82, the rate would have been 24.3 percent 
(21,045 graduates versus 86,463 Grade One students six years before). If Grade Seven 
admissions are used to measure overall progression for primary education, the progression 
rates would have been 23.5 percent (20,332 enrollees in Grade Seven in 1982/83 compared 
to the 86,463 Grade One students in 1976/77). Both progression rates have analytical value 
(although one based on graduation rate normally is preferred in measuring educational 
effectiveness), but one must be clear concerning the basis of rate before using it for policy 
analysis purposes. 

Some of the problems with repetition will reappear in the calculation of overall progres- 
sion rates. In the above examples, the number of graduates — exam passers — may include 
students who began in an earlier Grade One cohort than 1976/77. Also, the Grade Seven 
enrollment in 1982/83 may include students who are repeating Grade Seven and even some 
who delayed transition from Grade Six to Grade Seven for one or more years. The analyst 
must be able to approximate the reality behind these numbers or the calculated progression 
rates are only a mathematical and not a statistical exercise. Any two sets of numbers can 
be used to create fractions, percentages, and even correlations. The difference between 
mathematical methods and statistical methodology is that the latter requires that the sets of 
numbers be related to conceptually generated variables and that the numbers represent 
adequat<^^measurements of the underlying concepts. Since many educational decision- 
makers have neither the time nor, in some cases, the training to make methodological 
assessments, the analyst bears a special responsibility for methodological propriety. 

The use of tracer studies of specific student cohorts can be used to supplement the 
information gained from more aggregate enrollment data. In Table Five, the results of the 
special cohort tracer survey conducted by the Y AR's Educational Development Research 
Center (EDRC) arc compared with the inter-grade progression rates generated earlier from 
the single-year cross section and the two-year cohort comparisons. While one normally 
might assume that the implicit errors in the cross-sectional method would render that 
estimate least useful and that the long-term cohort approach used by the EDRC is the most 
sophisticated approach, the results in Table Five indicate that these two rates are more 
similar to one another than cither is to the two-year cohort rate. As noted earlier, one can 
be sure, in an expanding educational system, that the cross-sectional approach will 
underestimate actual progression levels. This is indicated in the table in that the cross- 
section rates arc the lowest of the three sets given. 
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But how can one explain the great difference between the two-year cohort data and that 
derived from the EREK: tracer study? Three probable explanations exist. First, the degree 
of enrollment over-reporting in the normal enrollment census may have increased (or the 
degree of under-reporting may have decreased) in recent years with the result that data from 
the two most recent years would be relatively biased upward. Second, the exclusion of 
repetition effects from the tracer study's calculation of progression inevitably lowered the 
progression rate. And third, it is quite probable that current progression rates are higher in 
the early grades than was the case when the early years of the tracer study were being 
conducted. In addition to the many expected reasons why progression rates might increase 
over time, the Y AR system was reducing its formerly large number of incomplete (less than 
six grade levels) schools during this time period. As schools added higher grade levels, 
progression rates increased because children could continue their primary education in 
their own community. 



TABLE nVE 

COMPARISON OF PROGRESSION RATES 
CALCULATED BY ALTERNATIVE METHODS 

TWO YEAR 

ONE YEAR INTER-GRADE SIX YEAR COHORT 



GRADE LEVEL 


CROSS-SECTION 


COHORTS 


PROGRESSION 


REPETITION 


GRADE ONE 


% 


% 


% 


8.2% 


GRADE TWO 


81.1% 


79.1% 


71.0% 


6.1% 


GRADE THREE 


79.9% 


88.9% 


74.7% 


12.0% 


GRADE FOUR 


68.5% 


90.0% 


72.3% 


7.4% 


GRADE HVE 


64.0% 


86.5% 


77.4% 


4.0% 


GRADE SIX 


68.2% 


91.6% 


81.3% 


5.4% 


GRADE ONE TO 
GRADE SIX 


19.4% 


50.1% 


24.1% 


% 



SOURCE: lEES Project, Yemen Arab Republic Education and Human Resource Sector 
Assessment, 1986. 
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This discussion indicates the need to understand the nature of the data with which one 
is working. A major use of progression rates as indicators of effectiveness might be to 
compare rates among schools, regions, types of control, gender of students, or some other 
characteristic. Proper policy interpretations of these comparisons can only be made if one 
knows the context of school operations in the various types of schools being compared. 

Psacharopoulos and Nguyen (1987) present an example of anotiier measure of attain- 
ment effectiveness: tiie age-efficiency indicator. The indicator illustrates Uie extent to 
which Uie actual age distribution differs from tiie official norm. For example, if the 
"official" age for primary school enrollments is from 6 to 12. a "gross enrollment ratio" can 
be calculated as follows: 

Enrollment at ages 5-17 
Gross enrollment ratio = 

Population at ages 6-12 

This can tiien be compared witii a "net enrollment ratio" tiiai is derived in similar fashion: 

Enrollment at ages 6-12 
Net enrollment ratio = 

Population at ages 6-12 

The age-efficiency indicator is derived from tiie ratio of tiie net enrollment ratio to Uie 
gross enrollment ratio. Because tiie population term cancels out. tiie age-efficiency 
indicator is equal to tiie ratio of age 6 to 12 enrollments to age 5-17 enrollments. The 
assumption is tiiat a system is more "efficient" when tiiere are fewer students outside tiie 
normal age for a level of schooling. This indicator, like many otiier enrollment/population 
ratios Uiat may be calculated, is useful only as long as cne is cognizant of tiie role of die 
educational system. Ifremediationisaprimary responsiblityfortiie system or somesubset 
of institutions, tiien tiie age-efficiency indicator could be lower in value and still indicate 
that the system or institutions were operating efficienUy. 

All of tiie attainment measures presented here are potentially appropriate indicators of 
educational effectiveness. When these attainment measures are combined witii otiier 
measures, such as tiiose of achievement and equity, an even better educational effective- 
ness indicator can be produced. And when tiiese effectiveness indicators are combined with 
cost data (generated by tiie interaction of inputs and process variables) one finally can 
establish an indicator of educational efficiency. 

B. Achievement Effects 

Achievement effects are perhaps tiie most commonly used of output measures. Test 
scores, either the absolute level or pre-test/post-test differences, are tiie most commonly 
used measures of achievement effects. However, tiie common use of testing, and its ready 
acceptance by educational decisionmakers, disguises a ratiier heated controversy among 
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educators and analystsconceming the psychometric properties of individual tests, the tests' 
relevance to desired educational outputs, and the definition of educational achievement as 
measured student change in terms of test results. 

The accepted credibility of testing as a measurement device appears to depend on four 
characteristics: (1) it is a seemingly objective measure; (2) test results lend themselves to 
inter-student and inter-group comparisons; (3) testing has been a traditional characteristic 
of educational systems and has been assumed to promote student discipline and effort; and 
(4) standardized testing can promote a centralization of educational authority. In contrast, 
the credibility of tests may be attacked on the basis of their validity and reliability relative 
to the underlying characteristics that the tests attempt to measure. 

In addition. Chapman and Windham (1986) point out nine generalizations that can be 
made about the use of a test score as an appropriate indicator of educational achievement: 

1 . Testing is not evaluating; 

2. Large scale standardized tests, the type most often used in developing countries, 
do not tell one what students know; 

3. Tests do not measure learning directly; 

4. Test scores are not perfect measures of knowledge or achievement; 

5. Often the domain of skills or knowledge that a test seeks to measure is pooriy 
understood; 

6. There are insufficient psychometric or social bases for establishing standards of 
test success or failure; 

7. As measured by tests, achievement may not be the most important criteria by 
which to judge an educational activity; 

8. Program quality cannot be improved simply by raising test performance stan- 
dards; and 

9. Gain scores should never be used as a sole basis for program evaluation. 

This general controversy over testing cannot be dealt with here; however, it is important 
that a more general recognition of the controversy over testing be promoted and an 
increased skepticism be encouraged toward the ready identification of test results with 
educational achievement 

However, some agreed measure of achievement must be established. This can be a test 
or the result of observational judgment (as it often is in the case of teacher assignment of 
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student grades). However established, the achievement measure for a single individual or 
group (and the group may vary in size from a number of students within a single class to 
the group of students in a national or multi-national classification) can be interpreted in 
effectiveness terms in six main ways: 

1. Absolute levc! of achievement; 

2. Average level or distribution of achievement; 

3. Group achievement relative to larger group average or distribution; 

4. "Mastery" level of achievement; 

5. Achievement gain; and 

6. Effect size. 

The absolute level of achievement is normally represented by a test score or assigned 
grade. If one understands the psychometric properties of the test or the mix of objective and 
subjective criteria used in assigning a grade, these measures have some value for policy 
interpretation. However, the absolute achievement measures rarely are used in policy 
analysis since it is more common that decisionmakers are dealing with groups of students 
and arc more interested in achievement relative to other groups or relative to a different 
time. 

The average level or distribution of achievement provides more information than the 
absolute levels in that one can now interpret individual level of achievement relative to a 
group average or pattern of distribution. While group means are the most commonly used 
measures of central tendency, in certain situations one may wish to use other measures such 
as the group median or mode. Alternatively individual scores may be slated in terms of the 
quartilc, decile, or percentile in which they fall relative to the full distribution. 

Similarly, the achievement of one group can be compared with that of a larger group. 
The comparison can be one of central tendencies or distributions. The latter might take the 
form of noting that 1 5 percent of one group scored above a certain level or grade while 25 
percent of the second group scored above the same level or grade. Since equity output 
concerns (discussed below) relate to distributional considerations, it is important that 
achievement data be available in terms of measures of the distribution and not just measures 
of central tendency.* 



* Postlethwaite (1987) supports the use of distributional comparisons and comparisons 
between pans of the distribution (e.g., quariile levels). As he noies (p. 157) "The 
question is, which knowlc^lge is useful and needed (even if it is sometimes not perceived 
to be needed)?" 
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In all of the cases discussed here it is important to know whether the assignment of a 
score or grade is norm-referenced or criierion-referenced. Norm-referenced achievement 
measures simply slate where an individual student's or group's achievement falls in the 
overall distribution of those tested or graded. Thus, a score of 90% or grade of "A" is 
interprelable only in a relative, not an absolute, manner. In fact, such scores are ordinal not 
cardinal in interpretation since one cannot state that a score of 40 represents one-half the 
level of learning indicated by a score of 80; neither can one assume that the achievement 
difference between an A and B grade is equal to that between a B and C grade. 

However, criterion-referenced scores (based on a criterion of skill or knowledge acqui- 
sition) also have their interpretive limitations. In addition to the standard psychometric 
problems of test design, Clark and Vogel (1985) have found that criterion- referenced tests 
have emphasized immediate educational outputs (practical knowledge of simple facts or 
routines) rather than the niore generalized and desirable outputs (the learning of concepts 
and principles). While such bias is not inherent in criterion referenced tests, efficiency 
analysis requires knowledge of the type of test or grade criteria used and appreciation for 
any significant measurement biases. 

The fourth form in which achievement measures may be interpreted is in terms of a 
"mastery" level. Here, the criterion-referenced test or grade is assigned a threshold value, 
below which it is judged that achievement is irrelevant in terms of mastery of the underlying 
criteria — whether ihey be conceptual or procedural (see, for example, Arlin, 1984, for a 
discussion of mastery learning and its relation to time and equity issues). Some mastery 
standards require that a score of 100 percent or grade of "A" be attained for mastery to be 
recognized. Other standards accept that mastery may be achieved at lower grades or scores 
and that achievement beyond mastery is possible. To specialists in the field of mastery 
learning these semantic distinctions are critical; to policy analysts it is necessary only to 
understand the level established for mastery and to be willing to accept the rationale for it. 

Mastery standards commonly are viewed as antithetical to norm-referencing. In fact, 
mastery learning standards may be viewed as the sine qua non of the criterion-referenced 
approach. Mastery tests are not only designed so that scores relate to underiying educa- 
tional criteria but also require that any judgment of the scores is not itself left to the norms 
or subjectivity of the analyst. 

The final two measures of achievement interpretation are achievement gain and effect 
size. Both measures are related to the economic concept of "valucradded." Thus, unlike 
the aforementioned measures of achievement, these measures imply attribution of the 
change or di f ference in ach ievement to some other change. In the case of achievement gain 
for a person or group, the explicit change is one of time. However, implicitly, the pre-tcst 
is prior to some educational event and the post-test is after it. The nature of the event can 
be simply a passage of time but more commonly it relates to some form of instructional 
intervention. For example, the intervention may be one year of school i ng or the use o f some 
educational material or alternative technology for a fixed period of time. Regardless, as 
suggested in item 9 of the list of generalizations about use of test scores to indicate 
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educational effectiveness, one faces a major question when using achievement gain as a 
measure of effectiveness: Namely, can one separate true achievement gains from 
anomalies introduced by test imperfections? 

The interpretive situation is improved when one is comparing gains in group means and 
improved even more if one uses residualized gain measures (in which the gain is 
residualized by regressing the pre-test on the post-test score), residualized true-score 
estimates (a statistical attempt to separate true scores from error effects and assess change 
only in iJie true scores), or an analysis of covariance (used when group not individual 
student gains are the focus and one can avoid the confounding of results because of non- 
random' constitution of the individual groups). Whatever approach is used, one sacrifices 
precision of measurement for comprehension by decisionmakers whose statistical training 
(and patience) may not be adequate to understand the modifications or even to understand 
why the modifications are made. Therefore it appears that, just as with testing generally, 
the use of gain scores, even with the above limitations, will continue to be acceptable to all 
but the most conservative methodologists. 

Similar problems exist in the use of effect sizes. Normally, effect size is defined as the 
difference between the average scores of an experimental and a control group, divided by 
the standard deviation of the control group. Effect size is a critically important concept 
since it often is the basis for deciding if a new instructional device or system deserves wider 
dissemination. 

In interpreting effect size measures as an indicator of effectiveness, three considerations 
must be taken into account. First, are the only differences between the control and 
experimental group those that are explicitly designed as part of the experiment? The 
infiuence of an experimenul condition on performance regardless of the nature of the 
experiment (Hawthorne effects) is a sufficiently pervasive phenomenon that some inherent 
skepticism toward experimental successes is justified. Anything that changes the normal 
routine and focuses special attention on teachers and students is likely to elicit improved 
performance. The methodological question is whether similar effects could not be 
achieved without the specific experimenul intervention that is being tested. 

Similarly, one must examine the control and experimental groups closely to see that the 
teachers and students in one group are not significantly different from those in the other. 
In large experiments this can be achieved by random assignment to the two groups; smaller 
experiments will require stratification and matching of critical determinants such as those 
discussed earlier under input and process measures (Kelly, 1983, 1984). Finally, one must 
attempt to guarantee that the effect size measures will reflect experimental effects and not 
differences in the quantity or quality of resources (the "greenhouse" effect). Too often, the 
**success" of experimenul classroom approaches compared to traditional models is a result 
of additional physical and human (especially supervisory) resources. In some poorer 
nations, the evidence that radio or television instruction or programmed learning is superior 
to traditional classroom results is hardly surprising if the traditional classroom lacks even 
the minimum teacher and insLructio^l jjialerial resources that the traditional approacli 
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presumes. Effect sizes generated from such flawed research has little value for policy 
unless it can show that the increased cost of the experimental alternative is better invested 
in the alternative than in improving the traditional classroom by investing the additional 
funds there. 

The second area of consideration in interpreting effect sizes is the presumed linkage of 
the test measure to the curriculum. The relationship of curriculum to testing is one that is 
mediated by classroom resources and behavior as indicated below: 
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Measured student performance, the basis for calculating effect size, can be determined 
by three major relationships: the relationship of curriculum expectations to classroom prac- 
tice, the relationship of performance measurement to classroom practice, and the relation- 
ship of performance measurement to curriculum expectations. The latter two relationships 
are the alternative criteria for examination design . Should the basis for test construction be 
the official curriculum expectations as stated in formal documents or the actual classroom 
practice observed in schools? Variation in measured student performance can signal varied 
classroom practice or can indicate a difference in determinacy in terms of the curriculum's 
effect on classroom versus the test instrurriCnt 

That this is a real source of confusion is indicated by the example of the Improved 
Efficiency of Learning (lEL) Project in Liberia. Early evaluation results, based on West 
African Examination Council tests that claimed to be based on the national curriculum, 
indicated no superior student performance in the EBL schools relative to traditional schools. 
However, on examinations based on the EEL instructional program, the EL schools did 
significantly better than the traditional schools. The interpretation of these contradictory 
results depends upon one's judgment as to the relative quality of the two curricula (the 
official national curriculum and the implicit lEL curriculum) and the validity and reliability 
of the two examinations. 

Ideally, effect size should be a function only of differences in classroom practice as 
indicated below: ACTUAL 
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If the examination designed to measure student performance is a valid and reliable 
device in terms of its relationship to the curriculum, then the effect size differences are 
meaningful; if it is not. then the analyst will have a difficult, if not impossible, task to 
disentangle the various relationships among expectations, practices, and measurement. 
Too often, examinations are designed to measure the curriculum that is implicit in the 
experimental practices rather than that officially documented by the educational authority. 
Where the official curriculum is not articulated with great specificity, then it can be 
impossible to determine effect size in any meaningful manner. In the lEL example, the lEL 
professional staff maintained that their examination (as well as their instructional system) 
was more closely related to the national curriculum than was the examination administered 
by the West African Examinations Council. Given the state of development of the Liberia 
curriculum at that time, the lEL assertion could not be readily rejected. 

Thus it may be seen that the calculation of effect size differences are only tlie beginning, 
not the concluding, step in the analysis of alternative classroom systems. As with the 
measurement of student achievement generally, effect size analysis is subject to a 
significant degree of debate and subjective judgment. As was noted in the introduction, 
effectiveness and efficiency analysis do not remove the need to make policy choices but 
they do have the potential to improve the basis upon which those choices are made. The 
degree to which this potential is realized will depend upon the care and sophistication witli 
which such data as achievement measures are generated and analyzed. 

C. Attitudinal/Behavioral Effects 

Output measures of educational effectiveness are dominated by the focus on the 
attainment and achievement issues discussed above. However, in many ways the public 
perception of education and the justification for government or community involvement in 
its regulation or financing are likely to relate as much to schooling's effects on student 
attitudes and behavior as it docs to the more easily quantified measures of attainment and 
achievement 

It is interesting to note that in the classroom assignment of grades to students, teachers 
in many educational systems are asked to consider student classroom behavior as a factor. 
Some classroom grading systems even include special categories for such items as 
motivation, behavior, discipline, effort, and citizenship. These same concepts are rarely 
translated into standardized measures used at a level of aggregation above that of the 
school. Among the reasons for this the most important appear to be uncertainly over the 
nature of the desired attitudes and behaviors, controversy over theability to measure these 
characteristics accurately, and confusion over how the classroom process relates to the 
development of the desired characteristics. 

The mix of desired characteristics can be expected to vary from nation to nation. The 
relative emphasis on independent performance versus group relationships, on competition 
versus cooperation, on religious faith versus tolerance of other beliefs, and on traditional 
versus modem values will depend both on the personal values of the nation's political 
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leadership (operating within constitutional limits ^A'here they exist and are operable) and 
these individuals' attitudes toward the propriety of using schools as a means of disseminat- 
ing their own views. Almost all nations will use education to promote national pride but 
in some this will be extended to the point of promoting support for the ruling party or even 
of a single individual or family. The degiee of controversy this promotes will depend upon 
the homogeneity of political views within the nation. Similarly, some nations have 
designed their education system specifically to promote the nationally-supported religion 
(or a form of secularism in those nations where religion is discouraged) while other nations 
use education to encourage diversity, understanding, and tolerance. It should be recog- 
nized that attitudes toward religion are rarely a democratically-determined phenomenon; 
the insistence on religious unity is sometimes most extreme in those nations where multiple 
religious beliefs are most competitive. 

Given that some agreement concerning desired attitudes and behaviors can be produced 
or imposed, one then must resolve the controversy over measurement. Problem? of validity 
and reliability are even greater in this case than in the measurement of achievement gains. 
As with the measurement of teacher attitudes, one must be concerned whether responses 
provided on survey instruments are either accurate or truthful given the tendency for 
respondents to be able to determine the socially preferred answer. Observation of behavior 
is time and labor intensive and still can be an imprecise means of imputing the anderlyirg 
attitudes or of predicting future behavior of students once they leave the restraints of ihe 
school environment 

A special area of attitudinal research has evolved from Inkeles' (1969) early work oa 
"modernity." Studies of modernity attempt to relate educational attainment or parti: ipa- 
tion to a set of attitudes that are stipulated to be more modem. Unfortunately, most studies 
of modernity appear more accurately to be studies of Westernization. In any case, the types 
of modem attitudes normally included in such studies are skepticism toward tradition, 
belief in science, and knowledge of the outside world. 

Table Six depicts a questionnaire used in a World Bank study of education in Tani^nia. 
While some of the questions might appear clearly related to modern attitudes, some relate 
to values about which even "modem" individuals could disagree. Because of doubt 
conceming causality, there is a stronger justification for using such modemity measures a;:; 
student inputs (determinants of student process or performance) rather than as educationa' 
outputs. As inputs, one also does not have to be as concemed with the issue of the soc .al 
desirability of the set of attitudes and can concentrate on whether the particular attuudes 
specified are identifiable and are positively or negatively correlated with desired patterns 
of classroom behavior or performance. 

Overall, a contradiction exists between thepossible importance of attitudinal and behav- 
ioral inputs of education and the serious difficulties of specifying, operationalizin^, and 
measuring these outputs. The contradiction that exists relative to these outpuu?,, however, 
is only another special case of the general situation for effectiveness analysis. Often, tb :^ 
more important and appropriate a concept or variable is, the more difficult it may be 

^ 63 



Chapter 4 



to specify, the greater the costs or barriers to operationalization. and the more substantial 
are the methodological limitations on measurement. To this general rule one can add that 
many of the most significant variables, once measured, are inlerpretabie only by subjective 
means. 



TABLE SIX 

MODERNITY QUESTIONNAIRE 

Please indicate your agreement or preference in the following statements by marking A for 
agree strongly. B for avTce. C for disagree. D for disagree strongly. 

If I was given a choice 20 shillings today or 40 shillings next month, 
I would take my 20 shillings today. 
It is usually better to meet familiar people than new people. 
Success depends more on luck than hard work. 
U is usually not wise to try new things. 

If you cannot solve a problem, the best thing to do is to leave it for a 
day or two. 

Good planning is more impoztant than hard work. 
Some people arc able to bring hann and misfortune to others through 
magic and sorcery. 
A child should plan his own future. 
Happiness is more important than success. 
The only people one can really trust arc one's family and relatives. 
There is no sense in woirying about the future. 
I would like to live in another country for some time. 
It is generally a waste of time to pSan for the future since unforeseen 
events can interfere with the plan. 

It is generally not possible to understand why people behave the way 
they do. 

Education is more importanl for boys than for girls. 
I always try to get better marks than my classmates. 
Often, feelings are a better guide to action than reason. 
I am more ambitious than most of my friends. 
It is better to learn about all nations rather than to concentrate on 
learning of one's own country only. 
One must plan each day for the next 

SOURCE: G. Psacharopoulos and W. Loxlcy, Diversification of Secondary School 
Curriculum Study, Guidebook (Washington, D.C.: Education Deparunent, 
The World Bank. February. 1982). pp. 14-15. 
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D. Equity Effects of Equality Measures 

The use of equity effects as measures of output differs from the use of the aforeoicn- 
tioned effects in that equity is 3 means of interpreting the other effects rather than an 
alternative, perse. Thus, equity effects can be expressed in terms of attainment measures, 
achievement measures, or attitude/behavior measures. Also, equity effects normally are 
expressed in terms of two dimensions: statistical measures of dispersion and measures of 
group differences. The first dimension of equity measures would include the range of a 
distribution, the quartile deviation, the mean deviation, the standard deviation, division 
among criteria levels, Lorenz curves, and Gini coefficients. The second dimension of 
equity would compare groups identified by such characteristics as gender, age, ethnicity or 
race, location, size of place, socioeconomic status, etc.) in terms of measures of the mean, 
mode, and median values as well as in terms of group differences in the values of the first 
dimension of equity measures. For example, one could compare mean achievement 
between males and females but also could compare the range of scores for the two groups. 
It is possible, for example, to have similar average achievement between male and female 
groups but to have male students achieve both the highest and lowest scores. Depending 
on the central tendency measure of achievement alone would disguise this phenomenon. 

It is important to emphasize that the statistical measures of dispersion* arc indicators 
only of inequality not inequity. Equity interpretations require subjective judgements con- 
cerning whether the inequalities are justified or acceptable. For the purposes of this pres- 
entation equity is best understood as denoting a judgment of "fairness" or **justice"; both 
of which are inherently subjective concepts. The measurement of educational output 
equality is important in two ways: equality is a basic indicator for making judgments of 
equity and the variation in output equality can affect student and teacher motivation. For 
students, one normally assumes that relatively high achievement promotes higher motiva- 
tion and low achievement results in the opposite. However, such is the complexity of 
human nature that, for some students, superior performance may lead to future compla- 
cency and poor performance at one point in time can be a goad to higher motivation for 
sixccss at a subsequent time. Whatever the conditions in individual students, the policy 
importance of equality measures arc that they are an immediate basis for assessing equity 
and u potential indicator of future motivation. 

All of the statistical measures discussed in tlie Appendix can play a role as indicators of 
educational effectiveness where equity considerations arc a policy issue. Obviously, these 
arc not the only measures used in the discussion of educational equality. In the next section, 
the discussion will review the second dimension of equity judgments that is based on com- 
paring groups (in terms of both central tendencies and variation) rather than measuring 
variation per se. Some examples of this second dimension have already been given. For 
example, one can contrast all of the disuibuiion measures mentioned above for specific 



* The statistical measures are dealt with in the Appendix to this monograph. 
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groups that are defined in terms of characteristics deemed important for policy. As noted 
earlier, the most important or commonly used of these characteristics are gender, age, 
ethnicity/race, location, size of place, and socioeconomic status. 

However, in addition to measures of distribution, inter-group comparisons can be made 
on the basis of the central tendency pleasures of mean, mode, and median. These terms are 
commonly understood but what is less well appreciated is when to use one rather than the 
others. As a standard rule: 

1. the mean should be used when each score should have equal weight, when 
maximum reliability is desired and when sUndard deviation or product-moment 
coefficients of correlation are to be calculated; 

2. the median should be used when an easily derived measure of central tendency is 
desired, when one wishes to neutralize the effect of extreme values, and when only 
ordinal rather than cardinal values of the distribution are available; and 

3. the mode should be used when a quick approximation of central tendency is 
desired and one has a special interest in m-aximum incidence (most frequently re- 
curring) values. 

A second problem with measures of central tendency used for ir«tcr-group comparisons 
is the degree of reliability that exists in terms of the differences between means, between 
medians, or between modes. The greatest statistical emphasis has been on the significance 
of differences between means. 

The issue is whether a measured difference in the means of two groups represents a resl 
and dependable difference or an accidental and transitory difference that is irrelevant for 
policy purposes. To assess significant differences in means th-. standard pi'xedure is to test 
the null hypothesis, that is, the hypothesis that no "true" difference exists. The test varies 
depending upon whether the means are themselves correlated or uncorrelated. The detail 
for conducting tests of this type can be found in any standard textbook on introductory 
statistics. 

In summarizing the use of group differences a<; an indicator of relati\e effectiveness, 
three conditions must be fulfilled before one is justified in making a policy inference: 

1. the difference between the measures of cen^ al tendency must be judged to be 
statistically significant; 

2. there must be some logical or statistical basis for assuming that the differences in 
central tendency and distributional measures are determined by factors within the 
control or infiuence of the school authonty; and 
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3. the magnitude of the difference is sucli that a policy emphasis on this condition 
may be shown to be cost-effective. 

If the final two conditions are not fulfilled, then the measures discussed here may 
indicate "real" effects but r^ot provide any basis for school reform. The policy alternative 
is cither to accept the effects as given or to consider new means of school operation 
(including variations in the quantity and mix of resources) that will allow the school to have 
an impact on these outputs. 

A final issue relative to measures of equity is that one must consider the school authori- 
ties' relative preference for different distributional paUems of achievement While few 
schools or school systems can be said overtly to seek inequality as an educational output, 
schools and school systems can be expected to vary dramatically in their tolerance for 
inequality. Some schools are interested primarily in increasing average achievement. The 
three means of doing this are (1) to attempt to increase achievement of all children; (2) to 
emphasize increasing the achievement of advanced students; and (3) to emphasize 
improving the scores of students that are below-average in achievement. 

Most teachers and administrators will assert Jiiat the first option is the one they pursue. 
However, if a school is judged in terms of its average achievement, the most rational 
procedure v/ould be to combine school resources with those students who have the greatest 
probability of increasing their measured achievement. Unfortunately, neither the research 
lilcraiure nor logic can provide an answer with certainty as to who these students are. 

Some teachers obviously believe that it is better or easier to invest their time with the 
more advantaged students; implicitly, they are making the assumption that the lack of in- 
telligerxc or motivation of the ixx)rer students cannot be overcome sufficiently to justify 
the teacher investing his or her time in these students. Other teachers operate in just the 
opposite fashion and assume the better students (by themselves or with educational 
materia^-s) can continue to do well and that the proper allocation of teacher time is in favor 
of the disadvantaged learners. 

If a different criterion than class or school average is used, then teacher behavior might 
change. For example, if a sch<X)l were judged by the proportion of students who pass a 
school leaving examination there would be an explicit incentive to concentrate resources 
on those students who are on the margin in terms of examination success. Alternatively, 
if the :chool were judged by its ability to avoid wastage (dropouts and repetition), the 
resources might be focused on the students who are on the margin in terms of minimally 
acceptable attainment 

Overall, judging schools by specific central tendency or distributional criteria is likely 
to creaie a **lriage" cflcct: 

1 . those students are arc judged to be successful without assistance may be left alone; 
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2. those students who are judged to be impossible to help with the amount of 
assistance available will be left alone, and thus. 

3. resources will be concentrated on those students who have the greatest marginal 
probability of being moved into the category of success. 

Multiple problems exist in this situation. First, no single measure is likely to be an 
appropriate indicator of educational effectiveness. Second, even if a single measure 
existed, there is no assurance that the political and bureaucratic environment within which 
education operates would lead to its identification and dissemination. And three, most 
teachers in developing nations simply do not have the skills to make the type of 
psychological appraisals of ability and motivation that are subsumed within a "triage" 
decision process. 

In the next section the discussion will move from outputs to outcomes of education. To 
repeat the distinction made earliei, educational outcomes are those effects that are more 
distant in time and more diffuse in incidence than are educational outputs. 

IL OUTCOME INDICATORS 

Dealing with educational outcomes involves the same two critical issues faced in dealing 
with educational outputs: identification and attribution. The issue of identification 
(including the steps of definition, specification, and measurement) of outcomes is similar 
in terms of relevance and difficulty to that dealt with in the outputs discussion. Although 
the variety of attributes to be included and the diversity of their incidence do make outcome 
identification slightly more difficult these are not insurmountable barriers. 

More challenging to the analyst is the issue of attribution, i.e.. causality and its direction 
between the education variables and the multiple variables that represent both alternative 
and complementary outcomes. As was stressed earlier, no strong consensus can be said to 
exist in terms of the degree of attribution of educational outputs to educational inputs and 
processes. The effect of non*school infiucnces and of relevant but unmeasured school 
influences, forces one to accept educational input-output studies with great care. Accep- 
tance of the assumed direction of causality is only one of a multiple set of assumptions one 
must posit before proceeding to alter inputs and processes in the hope of altering 
educational outputs. In dealing with outcomes determinacy. one must accept a number of 
assumptions and be satisfied with a lower degree of certainty bef ore proposi ng that a change 
in educational outputs can lead to a desired change in educational outcomes. 

The reason for the heightened uncertainty is that outcomes are the result of the interac- 
tion of educational outputs with a great variety of external influences. These external 
influences may include the determinants for admission to higher levels of education and 
training, the supply and demand conditions in the labor market, or the multitude of planned 
and accidental influences that shape an individuaPs attitudes and behavior. In summary, 
educational outcomes are determined by many other factors itian tlic nature and quantity 
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of educational outputs and the degree of determinacy of inputs to outcomes is certainly less 
than the determinacy of inputs to outputs. 

This discussion of educational outcomes will serve as the basis for ihe later discussion 
of external efficiency. The outcome measures that will be reviewed in detail here are the 
following: 

1. Admission to further education and training; 

2. Achievement in subsequent education and training; 

3. Employment; 

4. Earnings; 

5. Attitudes/Behaviors; and 

6. Externalities * 

Each of these outcome categories can be used as an indicator of effectiveness and, when 
combined with appropriate cost data, as a measure of external efficiency. 

A, Admission to Further Education and Training 

As one graduates from each level of training the two major alternatives that one faces 
are to seek immediate employment or to continue education and training. As the level of 
educational attainment increases, the opportunity cost of education also increases in terms 
of foregone opportunities for employment and earnings. Some students are able to engage 
in part-time employment while continuing their education and training activities; other 
students may be fortunate enough to receive an education or training stipend that helps 
compensate them for their lost earnings while in school. 

In both cases, the effect is to alter the benefit/cost relationship that is the basis for the 
schooling/training decision. In some situations, excessive subsidies can make continued 
education or training not only superior to the student's present alternatives but superior to 
the employment alternatives they face after graduation. In developing nations, this 
situation has occurred most frequently in teacher training programs and in vocational/ 
technical training activities. The provision for part-time employment and the granting of 
stipends must be reviewed periodically to assure that these special opportunities are an 
incentive for training and not an incentive to avoid or postpone graduation. 



* Externalities are, in fact, a classification of the incidence of the prior five categories of 
outcomes rather than an alternative outcome category (as will be clarified in the later 
discussion). 
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Just as the graduate must choose between employment and further education or training, 
so can the effectiveness of the curriculum and instructional activities be judged in terms of 
how well the graduate is prepared for these two alternatives (and, indeed, how well 
prepared the graduate is to make a rational choice between the two). Unfortunately, no 
consensus exists in terms of the relationship between the school factors (inputs, process, 
outputs) and these two outcome alternatives. The propensity of educational managers to 
** vocationalize" the curriculum whenever graduate employment becomes a problem would 
suggest that a clear relationship has been established between curriculum and employment. 
This "vocational school fallacy", as Foster (1965) has termed it, has persisted in the face 
of a variety of logical arguments and statistical analyses (e.g., seeChapman and Windham, 
x985). 

At the eaily stages of education it is assumed that literacy and numeracy skills are of 
substantial applicability in both the labor market and in the competition for access to further 
schooling. The further one moves along in the educational system, the more intuitively 
appealing it is to assume that the school should provide some skills directly related to 
immediate employment This tendency toward vocationalization is reinforced when 
severe bottlenecks are introduced into the system with the result that substantial portions 
of the age cohort are forced out of the academic system. Unfortunately, the benefits of 
training that are provided by vocational opportunities often are offset by the stigma of 
academic failure or ineligibility that employers identify with vocational school partici- 
pants. 

To use the students' progression to further education or training as a measure of 
educational effectiveness involves several dangers of misinterpretation. First, the choice 
tocontinue may be more a function of educational proximity than of past performance. The 
high rates of educational progression in urban areas are, in part, a function of better 
achievementbutalso are a function ofthe greater availability of further opportunities in the 
immediate area. An equally accomplished rural student may not proceed to further 
education or training simply because thedistance to a higher-level school may be too great. 
Even the provision of boarding schools at higher educational levels only reduces the cost 
differential, it docs not eliminate it. And boarding may raise other cultural and financial 
limitations that are disincentives for the rural student 

Second, admission standards may vary over time or among locations such that it is 
difficult to identify effectiveness with educational progression rates. It is best to view 
admission criteria as the product of a supply and demand situation in which the supply of 
places in the higher level of education or training are demanded by graduates from the 
prerequisite level of education. Where and when demand is high relative to the supply of 
places, admission standards may be increased. Where demand is relatively low, admission 
standards may have to be reduced so as to fill all places. 

The point is lliat academic standards for admission to a higher level of education or 
training are an interactive phenomenon incorporating aspects of both the supply of places 
for students and student demand for places. Thus, for any point in time, it is possibly 



70 



Indicators of Effectiveness: Outputs and Outcomes 



misleading to use progression rates alone as a direct indicator of past educational 
effectiveness. It also is important to remember that admission standards based on 
examination results alone measure only one of the multiple outputs of education. The value 
of progression rates as an indicator of educational effectiveness will be determined by one's 
subjective valuation of the criteria used in selecting students for further education and 
training opportunities. 

A third source of misinterpretation that can result from the use of progression rates as 
indicators of educational effectiveness is the problem of costs. This is, in fact, a more 
general case of the specific problem discussed above in the consideration of proximity as 
a determinant of further educational participation. The decision to continue one's 
education is not based solely on one's level of intellectual or social preparation; it is an 
investment decision that must consider costs as well as the probable benefits. And the 
perceived effect of costs will differ among individuals depending upon their family 
resources (assets and income) and the availability of financing (grants, loans, or work 
opportunities). 

Once again, the analyst may feci confident that progression rates are positively corre- 
lated with past educational effectiveness yet still be reluctant to use progression rates as 
indicators of relative effectiveness among classes, schools, or regions. For example, two 
schools could graduate students who. by all standards, are identical in their educational ac- 
complishments. One school may have a 70% progression rate to the next level of education 
and the other only a 25% rate. Since the graduates have the same educational background 
the difference in progression rates must originate from other determinants. Unless one can 
control for all of the differences other than educational quality that may determine the 
decision to continue education or training, there is no o.bVious justification for assuming a 
class, school, or region is superior in effectiveness Wfely on the basis of differential 
progression rates. 

The fourth and final major source of possible misinterpretation of progression rates 
relates to the differential value of further education. As noted above, the decision to 
continue schooling is an investment decision based on both costs and benefits. Even where 
educational effectiveness and costs are similar, students may face different probable 
benefits to further education. Education and training skills and knowledge are valuable 
depending upon their complementarity to other human capital characteristics of graduates 
and the nature of the labor market. The issue of complementarity is illustrated by the 
example of two graduates who differ only in terms of the business or professional 
associations of their families. The graduate with the advantage of these associations can 
expect a much shorter job search period and. probably, a higher initial salary and greater 
lifetime earnings. The graduate who is equivalent with the first except for these familial 
professional associations, must discount the benefits of education in terms of higher job 
search costs, lower initial salary, and lower lifetime earnings. At the margin, sucli personal 
differences may cause variations in progression rates that are totally unrelated to the 
effectiveness of the educational institution. Gender, race, and ethnicity are other human 
capital characteristics that may. in cases of employment and/or wage discrimination, or 



Chapter 4 



labor market segmentation, have differential degrees of complementarity to acquired 
academic and training skills. 

In addition to family advantages, a second major factor affecting probable benefits is the 
nature of the labor market itsel f . The difficulty and cost of transporlalion as well as possible 
segmentation of the market between modem and traditional enterprises have aggravated 
the extant differences promoted by the persistent division between urban and rural markets 
for employment. The result is that two graduates of identical educational skills may make 
different educational progression decisions and both graduates will have made a rational 
choice given the probable benefits they may expect 

These human capital and labor market differences can be attenuated by restrictions on 
discrimination, improved labor mobility, and greater access to information. In fact, 
unequal access to information may itselfcreateadifferentiatingeffecton progression rates 
in some categories of graduates (it will usually be the more rural and economically 
disadvantaged ones who also have the least and least accurate information). At times, dis- 
advantaged candidates fail to continue their education and training because they do not 
comprehend the probable net benefits or realize the actual availability of financing. More 
often the case is that the match between graduates and future opportunities is not a proper 
one because the appropriate information and counseling system does not exist. The failure 
of societies, developed and developing, to invest in such systems commensurate with their 
enormous investments in education is one of the major anomalies of the human resource 
sector. 

In summary, progression rates normally may be viewed as a positive correlate of 
educational effectiveness. However, relative effectiveness can be judged by using 
progression rates only when all of the other major determinants of the progression decision 
are controlled for adequately. 

B. Achievement in Subsequent Education and Training 

Since one purpose of education is to prepare the student for further learning, the use of 
measured achievement at a higher level of education or Gaining as an indicator of the 
effectiveness of education may appear an obvious choice. In fact, progression rates are 
much more commonly used than are future achievement measures. There arc three reasons 
for the infrequency with which future achievement measures are used: time delay, 
measurement problems, and uncertain determinacy. The time delay problem is obvious. 
If one must wait several months or even years to measure achievement at the next level of 
education or training then there is an inherent delay in being able to assess, analyze, and, 
if necessary, reform the earlier programs. While the conduct of education is an ongoing 
activity the attention spans of politicians, administrators, and even analysis are finite. The 
assessment of future achievement and the attempt to relate it to antecedent educational 
experiences is a valuable activity but not one that can satisfy the system's need for timely 
results. Because of this, future achievement analysis probably will continue to be used 
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(where it is used at all) as a complementary research acLiviiy to less time-extensive 
evaluations of educational effectiveness. 

In those cases where time is not a barrier, there are still a set of measurement problems 
related to the analysis of future achievement. All of the cautionary comments presented in 
the earlier discussion of achievement measures as outputs would apply as well to the use 
of future achievement measures as educational outcomes. In addition to the normal 
problem of assessing and interpreting differences in school grading or examinati.on results, 
one must also be concerned with the problem of changes in the group being measured. If 
one is able to trace the individual students this is less of a problem. However, as is more 
common, one may trace a group which itself can undergo changes. 

For example, if one wishes to assess the effectiveness of Primary School A graduates in 
terms of their achievement in Secondary School X, two problems exist. First, all School 
A graduates may not go to secondary school (or, because of migration, may go to a school 
other than School X). Second, the grades or examination results of School X may include 
the performance of students from schools other than School A. The result is that the 
achievement levels in Secondary School X are not solely related to the graduates of 
Primary School A and may not even be predominantly related to those graduates. 

This methodological problem unlike those raised earlier for assessing achievement 
results, is relatively easy to resoWe. It only requires that an explicit tracer study approach 
be adopted from the beginning. However, the problem is a real one and should preclude 
unjustified ascription of future achievement effects to a preceding level of education when 
one has not taken into account the constituency of the measured group. 

The final reason for the scarcity with which future achievement measures are used is the 
problem of determinacy. While most will concede that achievement at any level of 
education or training is determined in part by the skills and knowledge the student brings 
from prior education, there is no consensus about the degree of determinacy such prior 
experience has over achievement. For example, in measuring Grade Seven reading 
achievement, in addition to the students' ability at the start of Grade Seven (itself an 
imperfect proxy for the effectiveness of earlier education) one must consider the effect of 
Grade Seven inputs and processes as well as the continuing effects of nonschool determi- 
nants. 

The result is that, while a major goal of education at any level may be to prepare students 
to achieve more successfully at the next level, the measurement of future achievement is 
not a certain indicator of prior educational effectiveness. Only by controlling for other 
concurrent determinants can one be assured of a proper estimate of the effect of prior ex- 
periences. And, because of the problems of separating school and nonschool effects, even 
this measure of prior experiences is not an indicator solely of educational effectiveness. 

Even with the three problems of time delay, measurement difficulties, and uncertain 
determinacy, the measurement of future achievement can still play a role in assessing 
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educational effectiveness. However, it can be implemented effectively only in a tracer 
study approach that will allow for proper control of other influences on achievement and 
a stable definition of the group being measured. Even in this form, the future achievement 
measure is not adequate by itself to indicate effectiveness. To be most appropriate for 
analysis, it should be used as part of a set of multiple indicators of educational effectiveness. 

C. Employment 

To those students who do not continue their education, whether the discontinuance is by 
choice or not, the major consideration of educational effectiveness will be how well 
education has prepared them for employment. Here, the term employment is used to 
encompass the full range of activities from household chores, to casual self-employment, 
to informal and formal entrepreneurial enterprises, and to employment in the modem 
formal sector. Too often the discussion of educational effectiveness has been limited only 
to the last of these employment forms, i.e., whether a school leaver can obtain employment 
in the modern formal sector. However, much of education's effect may be revealed in the 
other forms of employment 

This especially is true for those who leave school at the primary and junior secondary 
levels. The most useful cognitive skills they will have acquired from the education system 
will be those of basic literacy and numeracy. Research, such as that conducted on farmer 
producuvity (Basu, 1969; Jamison and Lau, 1982; Jamison and Moock, 1984; CoUear, 
1986; and Chou and Lau, 1987), suggests that these basic educational skills can have a 
direct effect on the ability of workers to acquire and use information. While some debate 
exists over the means by which basic education translates into greater productivity (is it the 
skills, per se, or the auitude toward new information), it has been a generally accepted 
premise for educational development in the last decade that of all education and training 
al temativcs, basic educational development has the most direct and cost-effective relation- 
ship with general economic development (World Bank, 1980). 

The basic education provided to women can have a variety of positive influences even 
in the home. These range from improvement in time allocation and better health and 
consumer behavior to a more supportive attitude for family entrepreneurial activities and 
the education of children. Where the culture allows female participation in employment 
outside the home, the provision of basic education can have at least as dramatic an effect 
on productivity of women as of men. Since women often engage in the small scale 
enterprises (tourist crafts, herding, wcaving,brcwing, etc.) that provide a cash contribution 
lo the family's subsistence agrarian income, the educational effectiveness issue is of vital 
importance in this regard. Often, it is the women in the family who control the cash funds 
used to finance family contributions to educational costs. 

The skills of literacy and numeracy are essential for all small scale entrepreneurial 
activities. While some such activities may exist without educated participants, the 
entrepreneurial markets will never become regularized or equitable without the abilities 
implied by literacy and numeracy. The effectiveness of education can properly be 
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indicated by how well school leavers are prepared lo engage their acquired skills to meet 
the entrepreneurial opportunities that present themselves, even in the most rural and remote 
areas of developing nations. 

As an indicator of educational effectiveness, employment is, however, only a partial 
measure. Obviously, the analyst needs to know the type of job and the productivity of the 
school leaver in the job to assess the ful 1 effectiveness of education . However, employment 
rates still are commonly used measures in the assessment of an educational institution's or 
system's effect on the economy. 

The calculation of employment rates normally is done by dividing the number of 
employed workers by the size of the active labor force. The active labor force is defmcd 
as the sum of the employed workers and all others who are actively seeking employment. 
A problem with the employment index (or the unemployment index which equals one 
minus the employment index) is that neither the number nor the proportion of unemployed 
workers who are "actively" seeking em.ployment remains constant over time. For example, 
if there are 1 ,000,000 individuals in the active labor force and 900,000 are employed, then 
the employment index is 90%. If, because of economic improvements, another 50,000 
workers are employed, the employment index, normally will not increase to 95% (950,000/ 
1 ,000,000) as one might expect. Because more jobs are available, some individuals who 
were not actively seeking employment will begin to do so, thus increasing the size of the 
labor force. If the effect of 50,000 new jobs is to attract 25,000 new individuals into the 
labor force, then the new employment index will be 92.7% (950,000/1 ,025,000) rather than 
95%. A similar pattern occurs during periods of poor economic activity: as employment 
declines some labor force parucipants abandon hope of finding employment and leave the 
active labor force. The result of this phenomenon (which is a function of the definition of 
the employment rate) is that changes in the index of employment are less than proportional 
to changes in the index of economic activity. 

In most cases it will be preferable to use the level of employment or the change in 
employment rather than the index itself as an indicator of how well education is preparing 
school leavers for employment. However, since aggregate employment data covers such 
a w ide age range (usually 16 or 1 8 years to 65 or 70 years) and such data is often unavailable 
or unreliable,thebestmeansofstudying education's employmenteffcctislhrough data that 

concentrate on recent school leavers. While such data may sometimes be retrievable from 
aggregate employment statistics, the most useful data is that collected from tracer studies. 
The use of tracer studies allows more detailed collection on \iic personal characteristics of 
the school leavers and of determinant characteristics of the labor market. 

The analysis of tracer study data on the education-employment linkage can be summa- 
rized in terms of thrcx^ decision points: (1) the decision to continue or discontinue 
education; (2) the decision to accept immediately available employment or engage in job 
search behavior; and (3) the decision to accept a specific form of employment. None of 
these decisions are free; each is constrained, at least in part, by the decisions of others. For 
example, in a system of competitive admission for higher levels of education some students 
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will not qualify for the next level of education. Even if they have the desire to continue, 
the availability of private schooling and of the resources to finance a private schooling 
choice will determine if the student will be able to continue his or her education. 

Once the decision is nfiade to seek enfiploynfient, the individual school leaver must decide 
whether to accept employment of a type that is available immediately (if any is available) 
or to engage in a job search process in anticipation of finding employment that is more 
suited to his or her skills and interest. Job search behavior, like education itself, is an in- 
vestment activity for the individual. Thus, it is subject to a comparison of benefits and 
costs. The job search costs may be reduced if the school leaver can engage in some form 
of employment while seeking a more suitable job (in the same manner that students finance 
educational costs through part-time employment while in school). The willingness and 
ability to migrate (an investment process itselQ can increase both the potential costs and the 
potential benefits of the job search process. Migration will be engaged in whenever the 
probable net benefits of migration (including the emotional and practical considerations of 
separation from the family and home community) are considered positive. The job search 
process for an individual is facilitated by the availability of information and personal 
connections. In this, as in so much else, the urban and higher socioeconomic status 
individuals will have an initial advantage. 

Job search (measured in time and success of job acquisition— ideally, this can be 
"weighted" by the quality of the job procured) is a superior indicator of educational 
effectiveness than are simple employment rates. First, the job search measure emphasizes 
the current pattern of interaction between educational leavers and the job market. Second, 
increases in the length of the job search period are the first warning of labor market 
stagnaUon for a particular skill or type of school leaver. For example, employment rates 
of school leavers six months after the end of their education could be constant at 85 percent 
over a series of five successive cohorts. And yet, job search data for the five cohorts could 
reveal that each successi ve cohort has taken a longer period of time to attain that 8 5 percent 
employment figure. Changes in the quality of jobs and in the length of the job search 
process are early indicators of possible labor market problems for a particular type of school 
leaver. The changes may be the result of cyclical variation but could signal a long-term 
("secular") change in employment patterns. For this reason, the value of tracer studies can 
only be fully appreciated when the studies are conducted on a regular recurrent basis. 

The final decision in the job search process relates to selection of a job ofa certain type. 
Based on available information, the school leaver should /lelect a form of employment that 
will maximize the net benefits (the present value of the sum of benefits minus costs) over 
time. However, this assumes that the individual has adequate patience and resources to 
forego immediate benefits in some occupations for more substantial but delayed benefits 
in other occupations. Just as a highly trained graduate may need to spend a longer time 
seeking a job complementary to his or her skills, so the same graduate may have to be 
willing and able to spend the initial years of employment earning less than they could in 
alternative employment. As Becker (1964) has explained, employment that provides 
general skill training (training i^ouan benefit other employees) will have to be financed, 
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at least in part, by the worker. This financing normally will take the fomi of lower wages 
during the training period. 

The second major consideration (in addition to training opportunities) in selecting an 
occupation is to consider the monetary and nonmonetary aspects of the job. Earnings 
(which will be discussed in the next section) dominate most school leavers' considetadons 
but stability of employment is also of great importance. The attractiveness of government 
employment in many developing nations relates more to stability than to wage levels or 
social status (lEES. 1984C). In more sophisticated decision situations the school leaver 
must consider the balance between earnings and "hedonic*' wage difference.^.. Hedonic 
differences are the positive or negative nonmonetary aspects of employment that may lead 
the school leaver to accept a lower or demand a higher wage or salary, Hedonic differences 
sometimes are restricted to psychological factors such as job status or intrinsic pleasure but 
often are expanded to include all equalizing differences such as location , difficulty , and the 
nature of any job-related hazards. 

However defined, the importance of hedonic and cqualizmg diifereuces is that they 
suggest the impropriety of using general job categories or monetary earnings as a sole 
measure for ranking employment success. The quality of a job may relate to where one is 
located (national capital, regional center, or local village) and the value of earnings are 
affected both by the cost-of-living and the range of goods available for puichasc. 

In summary, employment rates, job search patterns, and job quality measures all are 
legitimate indicators of employment outcomes of education. None is a perfect measure 
when used alone but. when used in concert and over time so that paitems may be identified 
and tracked, they have a collective value as an indicator of educational effectiveness. Be- 
cause such collective measures are not reducible to a single numerical index, the tendency 
in policy analysis has been to use the employment measures separately or to ignore 
employment measures as distinct indicators in favor of the more easily understood measure 
of earnings. 

D. Earn"::gs* 

Of all measures used to indicate the effectiveness of education, the earnings measure is 
second only to achievement. Since the popularization of human capital models during the 



Economists distinguish earnings from income by linking the definition of earnings to ihe 
monetary benefits of employ ing the individuals human capacities (physical and menial) 
during the current time period. The latter condition is an important one because 
income is defined as inclusive of earnings plus "unearned" monetary receipts such as 
rcnu;, dividends, interest, and transfer payments. Since the basis for these latter receipts 
may he past earnings, they are "uneamcxl" only in icnns of this time period and !hc 
phrase is not intended to imply a judgment concerning lha propriety or ethics of 
receiving these forms of income. 
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1960s, the earnings measure has attained a consensus of acceptance among economists as 
a pnmary outcome measure for education. There are many reasons for this but the three 
most important appear to be the following: 

1. The logical and empirical obviousness of earnings as a goal of individual 
educational choice. Both statisucal surveys and the individual experiences of 
pohcy analysts suggest that an increasing majority of students are pursuing 
education as a means of increasing their personal economic advantage. It is 
accepted that earnings are the best signal to students of the economic advantages 
available in occupations and the best indicator of the success of graduates in 
procuring the desired advantages. 

2. Monetary earnings arc an undimensional numeraire. Unlike almost all other 
effectiveness indicators, earnings have a unit of measurement that appears to be 
readily understood by mosl people and to have a similar meaning across locations 
and time (allowing discounting for changes in the purchasing power of currency). 
Thus, the monetary measure of earnings is seen providing a "common yardstick" 
(Ray, 1984) in comparing different types of benefits and benefits with costs; 

3. Earnings figures are readily available. There is a common assumpUon that 
mdividuals are willing and able to report their earnings (as well as income) and 
that such reports are a reliable data base for analysis. Many education studies 
assume that students arc able to report on parental or total family earnings with a 
similar degree of accuracy. 

None of these three reasons for the popular use of earnings measures is fully refutable 
but neither is it necessary or reasonable to accept them without question. 

If one decides to use earnings measures as the primary outcome measure there remains 
a large set of residual decisions which must to be made. Among these, the most significant 
include the following: 

1 . to use initial earnings or lifetime casings; 

2. to use cohort versus cross-sectional data; 

3. to use group means versus median's versus marginal values for earnings; 

4. to attempt to control for ability differences in eamings determination; 

5. to use a wage function approach or to accept nominal wage differences as given; 

6. to correct for purchasing power and other equalizing differences; and 

7. to validate eamings reports. 



S 



Indicators of Effectiveness: Outputs and Outcomes 



Obviously, the rationale for investing in education is in terms of education's potential 
effect on earnings over the full lifetime of the educated persons. Based on this understand- 
ing, it would seem equally obvious that lifetime earnings are a superior measure to initial 
earnings as an indicator of educational effectiveness. The difficulty is that there are severe 
problems in forecasting the expected lifetime earnings of any particular individual or group 
at a given point in time. To use past earnings patterns for different levels and types of 
education or training is appropriate but these figures, even if available and acceptably 
accurate, must be modified to lake into account changing labor market conditions. 

Dore (1976) has presented the definitive explanation for the devaluing of educational 
credentials over time in both developed and developing nations. The problem is most 
dramatic in developing nations where the number of high level jobs is small and where 
educational expansion at the poslsecondary level is proceeding rapidly. Within a single 
generation an older sibling's college degree can provide entree into senior government 
service, a middle sibling's degree can qualify him or her for a director's position, and the 
youngest sibling may be fortunate, with exactly the same degree, to obtain an entry level 
clerical position in a government ministry. In this environment, the use o( ex post earnings 
data can greatly overstate future earnings potential. Such earnings data may indicate the 
effectiveness of education twenty or thirty years ago but is not a valid indicator of the 
current effectiveness of education. 

The value of initial earnings as an effectiveness indicator is that it provides an immedi- 
ate measure of education's interaction with the labor market There still are problems 
concerning the relationship of initial to lifetime earnings and of earnings as a result of 
education rather than hedonic or equalizing differences, but the initial earnings measure 
is often to be preferred to ex post earnings measures as an indicator of educational 
effectiveness. 

Figure Two indicates the difference in agc-eamings profiles depending upon one's use 
of cross-sectional or cohort profiles. As Colbcrg and Windham indicated in 1970, the ex 
post cohort and cross-sectional profiles each indicate quite different aspects of earnings 
patterns over one's lifetime.* 

The cross-sectional profile is useful to indicate the relative earnings of individuals of 
different ages, but »vith the same level of education, at a single point in time. The relative 
concavity of the curves can indicate a varying scarcity of educational qualifications among 
the age groups and/or changes in the cuality of education over time. The cohort profile 
traces a single age group over time and indicates how the single cohort's earnings adjust 
through the years to changes in the "vintage" of skills, continued on-the-job human capital 
investments, and different relative scarcities of educational qualifications. 



Sec Bowman (1986) for a current example of the imporumcc of considering cohort 
effects in the analysis of educational/earnings relationships. 
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FIGURE TWO 

COMPARISON OF CROSS-SECTIONAL 
AND COHORT EARNINGS PROHLE 




If one considers the age 25 group in 1960, one can sec ihai using the cross-sectional 
profile as an expected earnings profile would have understated the increase in earnings dra- 
matically over this group*s lifetime. Some of this difference in profiles would be reduced 
if one convened all earnings to 1960 purchasing power equivalents ("real" earnings 
profiles). Even with this adjustment, real productivity gains over lime would cause the 
cross-sectional profiles to understate the realized cohort profiles. 

One tneans of improving over the use of initial earnings alone as an educational effec- 
tiveness indicator is to take the ratio of initial earnings to lifetime earnings for the most 
recently available cohort. Modify this ratio by current forecasLs of changes in productivity 
and the future scarcity of the educational credential of the group under study . and then this 
new ratio can be applied to the current measure of initial earnings to produce an expected 
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lifetime income value. If this is appropriately discounted for time preference, one has a 
relatively simple approximation of education's exj)ected lifetime effect on earnings. This 
process assumes an acceptable quality of past earnings data and future forecasts of 
productivity, labor market changes, and infiation. For mos^t developing nations, the wisest 
decision may be to use initial earnings alone as the effectiveness indicator and involve the 
other data on earnings profiles, etc. in one's policy analysis but not in calculation of a 
quantified indicator. Finally , one always must return to the basic rule of the advantage for 
recurrent collection of data and designing all reforms and innovations with sufficient 
flexibility so that further changes may be made as more and/or better data become 
available. 

Another debate over methods of calculating earnings measures relates to the preference 
for the mean versus the median as the preferred measure of central tendency and of the 
relevance of either measure of central tendency compared to marginal earnings as a basis 
for estimating educational effects. There is no need here to repeat the common argument 
over the advantages of mean versus median values. It is adequate simply to remember that 
every advantage of mean versus median can equally be interpreted as a disadvantage. The 
choice of central tendency measure devolves to a question of how one wishes to deal with 
extreme values in the earnings distribution. If they are considered significant (as in calcu- 
lations of earnings probabilities) the mean earnings measure should be used; if not (as 
when data quality is assumed to be most questionable at the extremes of the range) then the 
median earnings measure is preferable. 

In terms of economic wage theory, the best earnings measure for estimating the effec- 
tiveness of education on the current age cohort would be marginal earnings, i.e., the 
earnings of the next worker or group of workers to be employed. This is a significant point, 
however, only if the marginal wage or salary is substantially different from the current 
mean or median. If the current mean earnings, for example, of a certain group of school 
leavers is $3,0(X) per year, the expected earnings of the next group to be employed would 
have to be significantly above or below this level for the marginal earnings consideration 
to be relevant. In competitive markets or where bureaucratic pay systems fix earnings over 
time, the mean/median earnings of the current group of employees and the marginal 
earnings of the next group will not differ significantly. 

The major exception to this is when an increased output of school leavers at a certain 
level exceeds tbc ability of the labor market to absorb them . In such a case one of two things 
can happen: earnings will decline below thatof past levels or else earnings will remain the 
same but employment probabilities will decline. The last is a point too often overlooked 
in earnings calculaiions as measures of educational effectiveness. The earnings measure 
used as an effectiveness indicator should not be the earnings level of employed graduates 
alone but the product of the probability of graduates being employed and the earnings 
level. 

For example, in the case above mean earnings were $3,000. If, for the next cohort, 
earnings remain the same but the percent of school leavers employed declines from 100 
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percent to 80 percent, then the effectiveness indicator should decUne from $3,000 (100% 
X $3,000) to $2,400 (80% x $3,000), The advantage of this definition of the earnings 
measure is that it can capture the effect of simultaneous changes in earnings and employ- 
ment probabilities. Thus, an increase in earnings to $3,100 could be offset by a decline in 
employment probability to 90%. The value of the indicator ($2,790) is less tlian the original 
$3,000 value even though nominal earnings have increased. Thus, the initial earnings 
measure discussed above should be understood to be a product of both earnings and em- 
ployment probabilities, 

Blaug (1972) has noted that there is no more consistent correlation in the social sciences 
than that between education and earnings. A major barrier to a su-aight-forward interpre- 
tation of this correlation as proof of causality is the issue of individual ability.* Theoreti- 
cally, it is possible for the correlation relationship between education and earnings to be 
spurious if both are, in facL functions solely of ability and thus unrelated to one another. 
An extreme view of the causal relationships would be as shown here: 




Educational 
Effects 



Personal 
Ability 




Earnings 
Effects 



As depicted here, education has no determinant effect on earnings. Personal ability is 
the determinant of the educational effects (outputs such as attainment and achievement) 
and of the earnings effects (normally considered an educational outcome). For this set of 
relationships to be valid, education can have no causal effect on earnings. 

This would mean thai any individual would have the same earnings regardless of 
whether or not they have any educational accomplishments. The correlation between 



* Ability as used here refers to the measured skills and knowledge possessed by individu- 
als at the time they begin a certain phase of education or training. It is not used in the 
sense of innate ability or to delineate genetic advantages or disadvantages. The 
difficulty of measuring ability in a meaningful way is great enough without attempting 
to parcel out original and acquired traits of students or trainees (a pursuit that is 
methodologically difficult and often irrelevant for the policy debate). Sec Griliches 
(1977) for indications of thedifncultics encountered in estimating ability effects in the 
education-earnings relationship. 
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education and earnings would be only a coincidence of culture or tradition (more able 
people both consume more education and earn more money) but earnings effects could not 
be used as an indicator of educational effectiveness because no direct causal link would 
exist between the two. 

Even if statistical evidence were lacking to support a residual effect of education on 
earnings when one controls for ability, logic and personal experience would lead one to 
reject the extreme model presented above. However, the extreme model contains more 
than a grain of truth. If one studies the consensus model of the relationships among ability, 
education, and earnings it is obvious that some consideration of ability effects must be 
taken into account in using earnings as an effectiveness indicator. 



In fact, two forms of ability dclcrminacy arc shown. A direct effect of ability on earnings 
and an indirect cffcx:i through personal ability's impact on educational outputs. 

In the earl icr analysis of educational outputs it was stressed that the effect of education 
in terms of achievement must consider the concept of learning gain or value-added even 
though these measures pose serious methodological problems. Similarly, the effect of 
education on earnings must be considered in terms of how much earnings for a group would 
have been with and without education. For example, assume two groups of students exist, 
A and B, and group A has greater skills and knowledge than group B. Further assume that 
the effect of upper secondary education on their earnings arc shown to be as follows: 

Group A increases average earnings from $3^00 to $4,500. 
Group B increases average earnings from $2,500 to $4,000. 

Group B will be seen to earn less than Group A both with and without the additional 
earnings. But the increase in earnings as a result of the additional education will be greater 
for Group B, whether measured in absolute terms ($1,500 to $1,000) or as a percentage 
increase (60.0 percent versus 28.6 percent). The point is that use of earnings levels alone 
would have indicated that the education of Group A was more effective because it would 
have compounded earnings and ability effects. 
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The example of personal ability is a special case of the more general need to control for 
non-education determinacy when using earnings as an indicator of educational effective- 
ness. The common means of this control is through use of an "earnings function" an 
equation that states earnings as a function of the influences that determine the supply and 
demand for labor. Just as the educational production function attempts to estimate the 
degree of determinacy of various inputs to educational outputs, the earnings function 
attempts to estimate the determinacy of all factors (including education) that influence 
personal or group earnings. The coefficient on education in the earnings function can then 
be used to indicate educational effectiveness. The use of earnings functions has been 
especially common in present value and rate of return studies (discussed later in the section 
on educational efficiency measures). 

This use of earnings functions can result in an example of apparent methodological 
precision disguising implicit methodological carelessness. In such statistically estimated 
functions there is no guarantee that adequate care will be taken to operationalize the 
education variable. A common definition of education is years of attainment; "more 
careful" studies may go so far as to include a variable to distinguish the type of education. 
Earnings function studies have varied widely in the care with which they have attempted 
to control for spurious relationships as well as the number and types of other determinant 
variables included in the earnings equation. The propensity of earnings to be determined 
by such educationally-correlated variables as personal ability, occupational experience, 
on-the-job training, and social advantage means that tlie failure to include such variables— 
and to operationalize them properly— <;an lead to a systematic overstatement of education's 
effect on earnings and thus will mean that earnings, as an indicator of educational 
effectiveness, will be a seriously Hawed measure. 

The argument here is not that the earnings functions are not perfect methodological 
instruments, the economist and educational analyst is in no position to establish such lofty 
standards Rather, the point is that earnings functions must meet a minimum standard of 
adequacy in producing a weight on the educatior-al variable that can be interpreted f^irectly 
as a measure of educational effectiveness in terms of the earnings outcome. 

Two final decisions are left to the analyst still prepared to use the earnings measure to 
indicate the effectiveness of education. The first is a methodological correction — adjusting 
for differences in the purchasing power of earnings in different settings— and the second 
is a data collection issue— validation of earnings reports. The four types of purchasing 
power adjustments most commonly made are for differences in the time of receipt of 
earnings, for urban versus rural differences, for regional c iffercnces, and for inter-country 
differences. 

Price differences between urban and rural areas and from region to region within a 
country arc as obvious as they are difficult to measure. The normal procedure to establish 
earnings equivalents is to identify a standard "market-basket" of goods and services and 
then to price this market-basket in the different locations. To do this successfully requires 
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a careful selection of a set of goods and services to be considered (this normally is based 
on a survey of consumer behavior) and the collection of accurate price data. 

One of the most erroneous perceptions one can have is that general prices are lower in 
rural areas of most developing areas. Prices for some locally produced foodstuffs may 
indeed be lower, but the large majority of modem products and services are more 
expensive. It requires less currency to live in rural areas not because prices are lower but 
because the levels of consumption are lower. An educational example can illustrate this 
point. Most educational systems have standardized pay for teachers regardless of where 
they are located. However, because of the common aversion to the isolation and hardship 
of rural life, the rural school consistently will have the greatest problems with late 
assignment of teachers, absenteeism, and turnover. Thus, the amount paid the rural teacher 
may be the same (although it can be even less because many systems place the newest or 
least-qualified teachers in rural schools), but the quality of educational service received is 
likely to be poorer. 

A similar situation may exist for textbooks whose prices are controlled by government. 
The rural parent may not have to pay more for a textbook but, because of economic realities 
such as transportation and storage costs, will find that fewer if any textbooks are available 
for their children. The rural parent often is left with the choice of no textbooks or buying 
a more expensive textbook copy on the "unofficial" market. 

A final example of confusion that enters into purchasing power comparisons is that of 
housing. Housing is frequently cited as the most dramatic example of why it costs more to 
live in urban areas. And yet, the comparison is not made with the same quality of housing 
in rural areas. Certainly it costs more to live in a modern house in an urban area (and es- 
pecially one with elecuicity and water) than in a traditional house in a rural area. For 
purchasing power comparisons to be meaningful, one must compare not the same nominal 
"thing'' but the same quality of thing between urban and rural areas and among regional 
locations. The current interest in teacher incentives research (Thiagarajan and Kemmerer, 
1987) is based in large part upon an increasing recognition that purchasing power and other 
disequalizing differences must be considered in teacher remuneration if any progress is to 
be made in providing similar educational services in dissimilar settings. 

The final earnings decision is one of validation. Probably the greatest single (non-tax) 
earnings validation effort ever undertaken is that currently engaged in by the United Slates 
government as part of the federally funded, need-based college student assistance program. 
While controversy persists over the accuracy of earnings reports and the efficiency of 
attempts to validate them (Windham, 1985), the main lesson that emerges is that the 
validity of earnings reporu? depends upon their use. Obviously, when self-reported earnings 
are a basis for the assessment of financial obligations (whether taxes oraparenlal or student 
contribution to educational costs), there is a vested interest in underreporting. Also, 
aggregate patterns of earnings distributions are more likely to be correct (because of the 
tendency of spurious errors to be partially offsetting) than are earnings reports on an 
individual basis (of the type used in many earnings functions studies). 
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Simply put, earnings reports are only as accurate as the care taken by the researcher to 
collect and interpret the data will allow ihem to be. At a mininnum, reports should be 
validated on a sample basis; this is especially important when students or school personnel 
are asked to estimate family earnings of the students. 

Once again, the detailed discussion of a variable such as earnings would seem to lead to 
discouragementconcerningitsuse. This is not the desired effect. Rather, the warnings and 
questions raised here are designed to promote care and reasoned use, not to promote 
abandonment of the earnings variable. In any case, the concern for overreaction is probably 
misplaced. Previous calls for greater methodological care in the economics of education 
have nor led to any excesses of methodological conservatism . The modal response has been 
to ignore the warnings and proceed as before. This pattern will not change until the users 
of data — planners and policymakers — are sufficiently informed to demand more careful 
analyses. As stated in the introduction, the promotion of that condition is the single most 
important goal of this report 

E. Attitudes and Behaviors* 

The attitudes and behaviors that are viewed as educational outcomes are basically the 
same as those discussed earlier as educational outputs. The ones which have received 
greatest attention in the research literature are attitudes and behaviors concerning educa- 
tion itself, toward social issues and understanding (an extension of the "modernity** 
concept), toward issues of human rights and responsibilities, toward political participation, 
and the effect of eu.^cation on consumption behavior. In this section, the discussion will 
focus on four of these: the effects on social responsibility, social views, political participation^d 
consumption behavior. 

A common claim made for education in the 1960s was that education had the potential 
to save substantial sums of public monies by reducing unemployment and propensities for 
anti-social behavior (specifically crimes against persons and property). The claim for 
unemployment reduction is in part valid and in part a fallacy of composition. To the extent 
that the education of individuals increases the social productivity of material capital, it is 
possible that increased educational programs can lead to an aggregate increase in the 
number of jobs. However, one can not conclude from the fact that unemployment rates 
decline among individuals as their level of education increases that increasing aggregate 
educational levels in the general population will have a direct and proportional effect on ag- 
gregate unemployment 

For example, college graduates, on average, have a lower unemployment rate than high 
school graduates. However, it is a nonsequitur to suggest that educating all current high 
school graduates to the college degree level would reduce their unemployment levels to 
those of current college graduates. Two problems exist with this scenario. First, other 
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social and personal differences exist between the current populations of college and high 
school graduates that will remain to the advantage of the current college group even if the 
educational advantage is removed (ability, motivation, and social class advantages are 
examples of these). Second, the scenario ignores that education serves a second purpose 
in addition to increasing productivity— it helps employers ration jobs among competing job 
seekers. 

Thus, if the difference in educational credentials is removed, employers may be 
expected to have even fewer traditionally college-level jobs than there are college 
graduates. The employers will need to devise a new or expanded system for rationing these 
jobs. Perhaps differences in the institutions attended will be used or individual tests of 
achievement will be considered. The result of removing the educational credential as a 
factor will be to the advantage of some of the former high school graduates. Rut this is the 
fallacy of composition in the original argument: increased education for a single individual 
helps because it can increase both the person's productivity and the scarcity value of their 
educational credentials. The latter effect will be lost if everyone's education is increased. 
The lesson to be understood from this discussion is that education Is a more immediate tool 
for reducing unemployment at the individual than at the aggregate level. Education can 
affect the lauer but aggregate unemployment, as discussed above, is an imperfect measure 
to indicate aggregate educational effectiveness. 

A second aspect of education* s effect on social responsib ility is the claim that education 
reduces crime (Ehrlich. 1975). To the extent that education can increase employment and 
earnings for an individual, and if crime is a result of individual poverty and need, the link 
between education and reduced crime may be established. However, at least two general 
modifications to this assertion may be proposed. The first is that the definition of crime 
normally is established by those with the highest levels of education and earnings and it is 
not surprising, therefore, that the greatest censure is reserved for those acts least commonly 
associated with this group. Second, a certain cynicism m^ght suggest that whatever the 
effect of education on the frequency of crime it certainly appears to have an effect on the 
scale and form of crime. Specifically, the corruption of senior private and public officials 
or the social disruptions that originate on university campuses, and, more generally, the 
phenomenon described in Western societies as "white collar" crime are evidence that 
education is an imperfect prophylactic against certain types of criminal behavior. 

Abandoning such cynicism, education may be seen to have two major effects relative to 
criminal behavior. First, education— in compact with the family and the religious 
institution— I. a major means tor any culture to inform its newest members of the 
definitions o; anti-social behaviors and the sanctions that may be imposed against them 
(Suaughan. 1982). Second.to the extent that education increases economic orcthical con- 
ditions of individuals, it raises the opportunity cost of criminal or unethical behavior. In 
summary, education docs have some effect on criminal propensity but the effect is not 
sufficiently direct or measurable that these behavioral outcomes can be used as indicators 
of general educational effectiveness. 

Q 

C w 87 




Chapter 4 



A more certain benefit of education appears to be the effect of increased education on 
fertility (Caldwell, 1980). G'ven the pressures placed on all social enterprises by the 
rapidly increasing populations, in the less developed nations, the ability of increased 
education to promote reduced population growth rates could be one of its most critical 
outcomes. The effect of education on fertility is the result of a complex process that 
involves other determinants such as income and urbanization (Cochrane, 1986) but 
basically involves both a change in attitudes toward family size and the ability to 
understand and utilize contraceptive techniques. Educational systems vary widely in terms 
of the explicilness with which population and fertility issues are dealt with in the 
curriculum. 

The second category of attitudinal and behavioral effects of education to be discussed 
is that of the social views of graduates or school leavers. To use such an effect as an 
outcome indicator would require that the social view or views be identifiable and 
m.easurable and that the causality of education be sufficiently certain that attribution could 
be assigned with an acceptable degree of confidence. An inherent problem with an 
outcome such as the social views of graduates or school leavers is that, in a politically 
diverse society, strong differences of opinion may exist as to the attractiveness of specific 
views. 

For example, the production in education of the acceptance of preferences for liberal or 
conservative political positions, religious or secular viewpoints, insular versus internation- 
alist attitudes, ;;tc,, will be commended or condemned depending upon the prevailing view 
of the current government or of the public. Nations vary dramatically in terms of the range 
of consensus views and in the nation's willingness to use education to propagate these 
views. The two most controversial areas are religion and politics. In certain societies there 
is no conuoversy (at least internally) in the use of education to promote the national religion 
or ideology or to make use of education as a vehicle to promote the current political lead- 
ership. 

One of the more impressive aspects of modem education, in both the developed and 
developing worlds, is that the same school curriculum can attempt to incorporate the 
conditional skepticism of the scientific method with calls for unquestioning acceptance of 
religious or political doctrine. The inherent conflict between the two curricular activities 
has led to a degree of political resistance to education in some countries where graduates 
have begun to apply rational tools to matters of religious or political faith. Political 
resistance to education, however, is constrained by the polar pressures of the social demand 
for educational opportunity and the economic demand for skilled graduates. In any case, 
social views, whether produced by education or not, will rarely be an acceptable indicator 
of educational effectiveness. 

A refinement of the preceding category has produced the behavioral measure related to 
political participation. This outcome of education is intended to be a more objectively 
measurable effect of education. There is little question that both political office and 
participation in electoral activity is positively correlated with educational attainment 
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This outcome of education may be classified appropriately as a social benefit of 
education; it may even be used as an outcome measure that is indicative of a specific aspect 
of the effectiveness of education; however, it deals with such a singular aspect of the total 
mix of desired educational outcomes that, by itself, it has only a minor role to play in the 
assessment of general educational effectiveness. More important in this regard is the effect 
of education on consumption behavior. 

In addition to the effect of education on earnings, research has long found an effect of 
education on how earnings are used to meet consumer needs (Michael, 1972). Basically 
education improves the ability of an individual to acquire information, to use information 
in making consumer choices, and, in concert with the educationally-influenced higher 
earnings level, to allocate consumption decisions in such a way as to increase individual 
utility over time. The first two effects, on tiie acquisition and use of information, arc 
identical in type to how education affects earning potential. Because of enhanced literacy, 
numeracy, logic, and knowledge (the experience of others) the educated person can acquire 
information from a wider variety of sources and at a lower cost than can the less educated 
individual. 

Once the information is obtained these same educationally acquired atuibulcs allow Uic 
educated person to process the information for better decisionmaking about consumer 
alternatives. The educated person will understand better the need to compare benefits and 
costs, will have more of Uic skills necessary to separate objective and subjective costs and 
effects, and will be better prepared to assign subjective evaluations to die decision process. 
Theodore Schultz (1975) has asserted that the major contribution of education to an 
individual's welfare is in education's ability to improve the individual's capacity to deal 
with "disequilibria." In tiiis context, the term "disequilibria" refers to any situation in 
which change is required and choices must be made. By improving the efficiency of both 
information acquisition and use, education enhances the individual's skill in improving his 
or h'tr own utility or happmess. 

The final influence of education on consumption behavior is a result of the interaction 
of the information effects discussed above witii the higher earnings. An often underesti- 
mated advantage of higher earnings is that it allows the more educated person additional 
resources to widen consumption choices across distance and over time. Urban studies of the 
poor often find higher unit costs for food and services because of die low income person's 
inability to travel to locations where prices are lower or to store items effectively and thus 
allow for savings due to purchases of larger quantities at one time. Also, die availability 
of credit and die educated individual's improved ability to understand and utilize it, permits 

a better planning of consumption over time. 

The reason why die change in consumption behavior normally is considered more 
important as an educational outcome than are the odier effects on attitudes and behavior is 
that the consumption effect is more objectively determ ined and, through its interaction with 
earnings, has a greater influence within die total set of outcomes. The latter point is 
important to understand because, regardless of die emphasis on methodological care and 
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detail, and the use of multiple measures and indicators espoused here, most effectiveness 
studies of education will concentrate on earnings as the primary outcome measure. 
However, if education increases the efficient use of as well as the production of, earnings, 
analysis who use only the earnings indicator will underestimate the relative effectiveness 
of education in terms of individual utility or happiness (the ultimate product desired for the 
investment in education). 

For example, if increasing education from the primary graduate level to the secondary 
graduate level increases mean earnings by $2,500 (when controlling for all other determi- 
nants of earnirgs), this amount will not depict the true difference between the two levels 
of education. If secondary school graduates are more efficient in using their income than 
are primary school graduates, again under ceteris paribus conditions, the "effect" of 
education will be in excess of iheeamings effect of $2,500. This implies that effectiveness 
or efficiency studies that use earnings effects alone as the measure of educational 
effectiveness will be biased consistently downward in their evaluations. 

This could lead to a less than optimal in vesunent in education by individuals or society. 
Also, since some types of education may have a greater effect on consumption patterns (and 
it remains to be proven that academic education differs significantly from vocational 
education in this regard) there could be a relative misinveslment among the different types 
of education and training if earnings levels alone are used as the measure of educational 
outcome effects. 

F, Externalities 

Externalities are not so much a separate outcome but rather a means of categorizing 
many of the outcomes already discussed here. An externality of education is any effect of 
^^ducation, positive or negative, on other individuals that was neither intended nor a basis 
for the education decision itself (thus, they are "external" to the process). The term 
"spillover" effects is sometimes used to imply the same lack of direct intention in causality. 
The externalities of education are the basis for identifying the "social" benefits and costs 
of education. While ideally such social effects should be available to all individuals 
without exclusion (see Windham, 1979B, for a discussion of this issue relative to higher 
education and the rationale for its inclusion in public or private financing decisions) the 
terms social benefits and social costs have now become, even to most economists, 
synonymous with positive and negative externalities, Windham (1972, 1976) lists eight 
major externalities of education: 

1. increased social mobility; 

2. change in the distribution of earnings or income; 

3. changes in altitudes and values; 

4. improved political participation and leadership; 
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5. lower unemployment; 

6. improved mix of manpower skills; 

7. enhancement of the productivity of physical capital; and 

8. an increased quantity and quality of research. 

In discussing research on these externalities in the case of higher education, Windham 
noted in 1972 that: 

The existence of beneficial externalities from the production of college 
graduates is uncertain; their manner and extent await elaboration even if 
they do exist; and if they are specified objectively, they still do not con- 
stitute by themselves, an adequate justification forpublic subsidies of the 
(higher) educational process. 

Fifteen years later this situation remains unchanged for higher education and, if 
anything, the uncertain existence of externalities and the question of their relevance has 
been extended to the earlier levels of education. 

Externalities enter into the analysis of educational effectiveness and efficiency because 
one must separate the determinants for judging the effect of education on the individual 
versus the effect on the society. Some outcomes discussed here, such as altered attitudes 
or values and increased political participation, are appropriate measures of educational 
effects only for society and not for the individual. By definition, externalities are those 
effects not considered by the individual who is making the education decision. 

However, these outcomes may be a legitimate measure to be included in society*s 
judgment of educational effectiveness. The use of the externality concept to categorize 
outcome variables between those relevant to the individual and those relevant only to the 
societal collective is crucial since effectiveness or efficiency studies are designed for 
evaluation and improvement of the decision process of individuals and of society. Most 
outcome research suggests that greater attention needs to be paid both to the improved 
identification and measurement of externalities and to a more careful separation of 
individual and external outcomes of the education process. 

Thisextended discussion of outcomesconcludes the discussion of educadonaleffective- 
ness indicators. As noted at the beginning of the outcome section, the measurement of 
outcome effects and their attribution in whole or in part to educational determinants are 
even more severe challenges than was the case for the more direct and immediate 
educational outputs. However, the outcomes hi.^e greater influence and scope in terms of 
capturing education's long term effect on individual and social utility. 
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In the next section, the discussion will turn to educational efficiency. This requires a 
combination of the effectiveness measures with information on educational costs. Follow- 
ing a brief discussion of cost identification and measurement, the efficiency analysis will 
be presented based upon the decision criteria models discussed earlier:^ benefit/cost 
analysis, cost-effectiveness analysis, least-cost analysis, and cost-utility analysis. Effi- 
ciency models will be studied in terms of single time period analyses but the discussion also 
will stress educational decisions where the incidence of effects and costs takes place over 
multiple time periods. 
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THE ANALYSIS OF INDICATORS OF EDUCATIONAL EFFICIENCY 



Efficiency analysis of education incorporates all of the concepts and issues presented in 
the preceding chapters on educational effectiveness. In addition, it adds the consideration 
of educational costs and imposes on the cost and effectiveness measuies specific decision 
models for the relating of efficiency measures to educational decisionmaking. It is 
critically important that one realize that the efficiency analysis can be no better than the 
effectiveness and cost data it incorporates. And data quality in this context refers not just 
to accuracy and timeliness but also to the scope and relevance of the effectiveness and cost 
measures used. 

The discussion of educational efficiency will begin with an introduction to the basic 
concepts of cost. In the earlier conceptual discussion brief mention was made of cost 
definitions (average, total, marginal, etc.). Here, the emphasis will be upon the operation- 
alization of cost concepts as measures to be used in the discussion of educational efficiency. 

L THE MEASUREMENT AND INTERPRETATION OF EDUCATIONAL COSTS 

In measuring educational costs for project or program analysis there are two main 
approaches that may be used: the aggregate approach and the ingredients approach. In the 
aggregate approach one uses cost data that already exists to estimate the quantity and value 
of resources used in the production of the educational outputs or outcomes under study. For 
example, in the lEES Project's Yemen Arab Republic (Y.A.R.) education and human 
resources sector assessment of 1984. government expenditure data were available for each 
major level and type of education. The cost data included expenditures on teachers and 
educational materials and some recurrent facilities expenditures (for major repair and 
maintenance acdvities). Also available were the central and regional administrative costs 
of the education system although the government normally did not divide these cost by 
levels and types of education. This division was accomplished by assuming that the share 
of administrative costs borne by each level and type of educaUon was proporUonai to 
enrollment. While an acceptable approach, the assumption of enrollment proportionality 
of costs probably overestimates cost at lower relative to higher educational levels. Most 
government administrators in the Yemen Arab Republic felt that there was a higher 
involvement of administrative resources per student at the post-primary levels of educa- 
tion. 

Unlike the accounting system in many nations, the educational accounts in the Y.A.R. 
also allowed inclusion of donor support for current expenditures (in the recurrent budget 
this was almost exclusively support for teachers). Even with the detail available on 
expenditure by level and type of education, the govemmcntof the Y.A.R. sUU realizes that 
much in its cost data system needs to be reformed and is in the process of making such 
improvements. The major data concerns of the government are general accuracy, diffcr- 
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ences between amounts allocated and actually expended for a givei; year, and the lack of 
adequate statistics on private and community contributions. 

The last aspect was dealt with in Botswana by using the ingredients approach, the second 
major costing method. In the ingredients approach one takes the separate categories of 
inputs (ingredients) and sums these to a total. Depending upon available date, the inputs 
may be summed on an aggregate (total expenditures) or unit (per student, classroom, or 
school) basis. In Botswana, tl.e calculation was done on a per-student basis, for each of 
three school control categories, with the separate inputs of the parents and family identified 
as follows: 

- initial enrollment contributions 

- registration fee for school leaving examination 

- student activities fee 

- uniforms 

- labor for school maintenance and repair. 

These private contributions varied in amount by level and type of education. At the 
upper secondary level and in vocational programs some schools also charged fees for 
materials and laboratory expenses. It should be noted that items with a life of more than 
one year — such as school uniforms — were annualized by dividing the original costs by the 
expected number of years of usable life. Table Seven summarizes the cost data calculated 
for the Botswana sector assessmenu 

A more extended examp!:; of the challenge faced in deriving cost estimates by the 
ingredients approach is prssented by the EHR sector assessment update report for the 
Govemment of Botswana (DEES, 1986B). Again, litUe expenditure detail was available for 
specific levels of the system for the major inputs (staff, equipment, materials, and 
facilities). Rather, the govemment budget (as indicated in Table Eight), presented 
expenditure categories including both activity forms (cen:ral administration — called 
"headquarters" — and curriculum development, for example) and levels of the system 
(primary education, secondary education, etc.) Unfortunately, the major school expendi- 
ture, teachers' salaries, was not divided by the level or type of education but instead was 
aggregated into the single category of the unified teaching services. Similarly, bursaries for 
all progrrjn types and levels were a single category and not pi ^sented so as to allow analysis 
of bursaries in specific programs. 

The solution to this was to develop, with the Botswana Ministry of Education personnel, 
a system for allocating the amounts in the aggregated categories across the levels and types 
of education. For teachers this was done based upon separate data on teacher assignments 
by types of academic and teacher training credentials. From this it was possible to derive 
the probable salary- levels. For other categories, such as curriculum development, adequate 
data did not exist for allocation 3nd, thus the category was folded in with other general 
administrative costs and distributed based upon the assumption of enrollment proportion- 
ality as was discussed above. 
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TABLE SEVEN 



BOTSWANA SECONDARY SCHOOLING 
UNIT COSTS. 1982-83 



Type 



Government 
Student Gov't Total 
Cost Cost 



Government-Aided Community Junior 

Student Gov*t ToUl Suidcni Gov*t Toul 
Cost Cost Cost Cost 



Tuition 

Uniforms 

Books/ 
Supplies 

Exam Fees 

Dinner 
(Day Pupils) 
or 

Room/Board 
(boarders) 

Teacher Salaries 

Other Salaries 

Other Expenses 

Subtotal 



P20 
10 



Boarders 



P20 
10 

35 
5 



35 

5 

12 76 

60 114 

357 
100 
238 



809 less 

tuition (boarders) 
or 

771 less 

tuition (day pupils) 

130 789 919 
82 751 833 



70 415 



P200 P20 (Grant) 
10 

Incl. in tuition 
5 



114 



70 395 465 



215 134 349 



SOURCE: lEES Project. Botswana Education and Human Resources Sector Assess- 
ment, 1984. 
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TABLE EIGHT 

GOVERNMENT OF BOTSWANA 
MINISTRY OF EDUCATION RECURRENT BUDGET 
1984/85 AND 1990/91 

1984/85 1990/91 

ANNUAL 



Department 


Expenditure 
(P'OOO) 


Percent 


Expenditure 
(P'OOO) 


Percent 


GROWTH 
RATE 


Headquarters* 


7.71 :- 


10.4% 


17,477 


12.6% 


14.5% 


Primary 
Education 


842 


1.1% 


1,083 


0.8% 


4.3% 


Secondary 
Education 


10,357 


13.9% 


23,094 


16.6% 


14.3% 


Teacher 












Training 


1.602 


2.1% 


2,404 


1.7% 


7.0% 














Education 


2.259 


3.0% 


9,241 


6.6% 


26.5% 


Nonfonmal 












Education 


817 


1.1% 


868 


0.6% 


1.0% 


Unified 












Teaching 
Services 


41,852 


56.0% 


67,528 


48.6% 


8.3% 


Curriculum 












Development 


1,032 


1.4% 


1,095 


0.8% 


1.0% 


Bursaries 


8,232 


11.0% 


16,249 


11.7% 


12.0% 


TOTAL 


74,749 


100.0% 


139,039 


100.0% 


10.9% 



♦Includes the University of Botswana and the Brigades 

NOTE: 1984/85 figures are forecasts and 1990/9 1 are National Development Plan 

6 targets. 

SOURCE: [EES Project, Botswana Education and Human Resources Sector Assess- 
ment Update, 1986. 
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The cost situation was made more complicated in Botswana by the existence of 
educational fmancial support from ministries other than the Ministry of Education, local 
support, and some private family costs. Based on available data and assumptions about 
expenditure levels and incidence, estimates were made for seven major levels or types of 
education (with additional detail on the three major vocational/technical programs). 

Table Nine presents the cost data for Botswana for the years 1983/84 (calculated for ih ^. 
original sector assessment— lEES, 1984B) and 1984/85 (government figures used in the 
assessment update^IEES, 1 986B). Differences between the two sets of figures result from 
real changes in the interim (for example, the government was making substantial new 
investments in vocational and technical education during this period), changes in enroll- 
ments (since these are unit not aggregate costs), better cost data, and different assumptions 
for distributing central costs. The latter two changes were initiated in response to some of 
the data questions raised in the initial Botswana assessment and the resultant reinforced 
appreciation for the value of accurate cos t data in policy analysis. While the average reader 
may be surprised by the disparity between the two estimates in some items, most cost 
analy S!5J would be gratified by the fact that the new figures follow basically the same pattern 
as the earlier ones. This especially is so given that the earlier estimates were derived under 
less than ideal conditions of time and resources. Like all effectiveness or efficiency work, 
cost analysis should be done on a recurrent basis, ideally on at least an annual basis, both 
to identify trends and to allow for a cumulative improvement in the methodology of 
deriving costs. 

Cost analysis has as its primary purpose the production of cost figures for use with 
effectiveness measures to produce indicators of educational efficiency. However, at times 
cost data are used without parallel effectiveness data and yet some conclusions, or at least 
inferences, concerning efficiency may be drawn. This is justified only in selected 
situations. For example, the data presented in Table Nine can be adapted to create a form 
of cost index. The most common approach is to set primary education equal to l.O and 
derive indices for the other levels and types of education in terms of their costs relative 
to that of primary education. For the 1 983/84 data inTableNine, the primary education cost 
is 189 Pula, so all other costs would be divided by this amount to produce the indices as 
shown below: 



Primary Education 1 0 

Secondary Education 4.0 

Brigades 5.3 

Other VocationalyTechnical 1 1.2 

Botswana Polytechnic 15.6 

Automobile Training Trade School 12.8 

Botswana Institute of Administration 5.2 

and Commerce 

Teacher Education 5.0 

University of Botswana 37.8 
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TABLE NINE 

GOVERNMENT OF BOTSWANA 
UNIT COSTS OF EDUCATION 
1983/84 - 1984/85 
(Pula per Year) 

ITEM 1983/84 1984/85 



Primary Education 189 160 

Secondary Education * 833 733 

Brigades 1000 1000 

Other Vocational/Technical 2123 3 160 

Botswana Polytechnic 2955 N/A 

Automobile Training Trade School 2428 N/A 
Botswana Institute of 

Administration and Commerce 987 N/A 

Teacher Education 938 1455 

University of Botswana 7143 8079 

Nonformal Education N/A 25 
♦Government or government-aided schools only 



SOURCE: EES Project. Botsymna Education and Human Resources Sector Assessment 
Update, 1986. 

A danger of diis form of cost analysis is that the politician or administrator may be 
unduly surprised at the scale of some of the differences. This problem can be aggravated 
if the data is presented in a polemic form such as "every college student means the sacrifice 
of places for 38 primary school students." The "sacrifice" measure is accurate only if the 
unit costs also are marginal costs (otherwise it may be possible to expand the number of 
either college or primary students without requiring a reduction in the other propoitional 
to the index) and budget levels are constrained for aggregate educational expenditure (a 
condition that unfortunately is increasingly common in all nations). 
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Obviously, costs differences of the type depicted in the indices must be offset by 
differences in the effecUveness of the educaUon and training categories. The cost data, used 
in isolation of effecUveness data, may raise important questions but can never answer 
them. The antipathy irany administrators and project directors have toward cost analysis 
is that unless such analysis is done for all compeung forms of educaUon, cost data can place 
a pro^m or project at a political or bureaucraUc disadvantage. This results because 
normal accounUng procedures in ministries often exclude some costs and overlook others. 
Thus a detailed analysis that reveals these costs will make a program or project appear 
relau'vely more expensive than other programs that have not been analyzed in a similar 
fashion Such comparisons of costs are best done when effecUveness among the altema- 
Uves is Uie same (or is assumed to be Uie same as in most least-cost analyses). This often 
is the case in pre-project assessments and even in some project evaluaUons. 

One of the most detailed analyses of costs done on a major educaUonal development 
project wasUiat conducted wiUi World Bank financing forthejointGovemmentof Libena- 
USAIDImprovingtheEfnciencyofLeaming(IEL)project(Windham,1983A,B,C,D3). 
These analyses covered issues of unit costs by grade level, cost variaUon among school 
locations projecUons of dissemination costs by altemaUve disseminaUon schemes, com- 
parisons of cost versus effecUveness between Uie EL and control schools, and a revised 
cost analysis based on changes in Uic materials requirement for Uie lEL program. 

The changes were encouraged in part by findings in Uie earlier cost analyses. Among 
the major points of Uie cost analysis were: (1) Uiat Uie lEL system had large economy of 
scale effects but that actual class sizes except at Uie grade one or two level did not allow 
Uie system to take advantage of Uiosc scale effects; (2) Uie system was more adaptable in 
terms of relative costs to urban schools Uian Uie rural ones for which Uiey were first 
designed ; and (3) that dissemination costs would be greatest in Uie rural schools because of 
high transport and management costs. 

Even wiUi Uiese concerns, Uie results of Uie cost analysis were posiUve for Uie lEL 
methodology Table Ten summarizes the cost comparisons for Uie original and revised EL 
system and for Uiree assumpUons about textbook costs. The two major residual concerns 
were the absorptive fiscal capacity of the Liberian government and Uie quesUon of Uie 
government's commitment to Uie lEL approach versus Uie tradiUonal textbook^based 
system In 1986 UieGovcmmentofLibcria,assistcdbytheUSAID-financedmESProjec-, 
designed an integrated lEL-tcxtbook system for consideraUon as Uie core mstiucuonal 



The lEL project involved the use of programmed teaching materials in Grades One and 
Twoand in UiefirstsemesterofGrade Three. Thereafter,programmcdlcamingmodules 

were provided for Uie remainder of Uie six yea- primary cycle. The modularized lEL 
system involved Uie use of basic instrucUonal modules, reading booklets, review 
booklets practice booklets and semester tests. At Uie higher grade levels student gmdes. 
test booklets and test answer keys, block tests, and an arts and craft manual were 
provided. 
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1 ABLE TEN 

RELATIVE PER-STUDENT COSTS OF INSTRUCTIONAL MATERIALS 



Grade Level and 
Instructional Material 


20 


NUMBER OF STUDENTS IN CLASS 

40 60 


Grade One 








IbL ((Jngrnal) 


$5.91 


$2.95 


$1.97 




3.55 


1.78 


L18 


iCXlDOOK ^A.^ 


9.38 


9.38 


9.38 


1 CXlDOOK \)^) 


4.69 


4.69 


4.69 


\ CXlDOOK (L-j 


2.35 


2.35 


2.35 


Grade Two 








IbL ((Jngmal) 


$6.27 


$3.13 


$2.09 


TPI ^D^'\/^c#»^^^ 
iCLw> ^Kcvisca^ 


4.13 


2.06 


1.38 


1 CXtbOOK (Aj 


10.88 


10.88 


10.88 


1 exiDOOK y^ti) 


5.44 


5.44 


5.44 


1 CXlDOOK ^V- ) 


2J7. 


2.72 


2.72 


Grade Three 








IbL (wngmal; 


S6.1 1 


$3.07 


$2.04 


iiZii^ ^ixcvisea^ 


3.34 


1.67 


1.11 


1 eXlD<X)K \/\) 


10.78 


10.73 


10.73 


TovtKrw^lr 
1CaI(XX/K v.^^ 


5.3/ 


5.37 


5.37 




Z.OO 


2.68 


2.68 


vjiduc rour 










S6.70 


$3.35 


$2.23 




Z.oU 


1.40 


.98 


ICaIDOOK \r\) 


9.42 


9.42 


9.42 


1 eXlDOOK ) 


4. /I 


4.71 


4.71 


ICAklAXJK 


Z.3o 


2.36 


2.36 


Grade Five 








lEL (Original) 


S6.70 


$3.35 


$? 23 


lEL (Revised) 


2.80 


1.40 


.93 


Textbook (A) 


11.65 


11.65 


11.65 


Textbook (B) 


5.83 


5.83 


5.83 


Textbook (C) 


2.91 


2.91 


2.91 


Grade Six 








lEL (Original) 


$6.70 


$3.35 


$2.23 


lEL (Revised) 


2.80 


1.40 


.93 


Textb<x)k (A) 


9.02 


9.02 


9.02 


Textb(K)k (B) 


4.51 


4.51 


4.51 


Textb<x)k (C) 


2.26 


2.26 


2.26 



SOURCE: D.Ni'. Windham, "Cost Estimates at the Revised 'Improved Efficient y of 
Leaning* Project*s Materials Component " 1983. 

NOTE: Textbook (A) refers to approved texts at one text ifCt studenL Textbook (B) 
refers to reduced price texts atone text per student. Textbook (C) refers to re- 
duced price texts at two per student 
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system for primary education in Liberia (lEES, 1986). Much of this design work was based 
upon updating and modification of the original cost analyses. 

Once cost estimates are generated, by either the aggregate or the ingredients approach, 
one still has to be prepared for the problem of their appropriate analysis. The relationship 
of costs to class or school size poses a special problem in this regard. In the lEL example, 
it was noted that the costliness of the EEL materials, and thus their relative cost when 
compared to textbooks, varied depending upon class size. When any input cost is fixed — 
that is, it does not increase in aggregate amount with enrollment, then the unit (per-student) 
cost of that input must decline as enrollment increases. 

The view of the teacher expense as a fixed cost of education presents a basis for 
examining this concept of unit cost. Part A of Figure Three indicates thai the average cost 
pcr-student has a declining (and asymptotic) relationship with enrollment when teacher 
cost is the only classroom cost. If one ignores the effect of class size or. student 
achievement, then one can see that smaller classes inevitably will have the disadvantage of 
a higher unit cost than will larger classes. The decline in average cost that comes with 
expansion in the size (scale) of classroom operation is an example of an economy of scale. 
Part B of Figure Three indicates the "step-function" that exists as one expands the analysis 
of unit cost from the single to multiple classroom situations. The peaks in the function that 
occur at points A and B result when a new teacher (or teacher plus classroom) is added to 
accommodate more students. If one has one teacher that costs $2,500 per year and fifty 
students, the unit cost is S50 ($2,500 divided by 50) as indicated at point A. If a new teacher 
is hired when the fifty-first student is granted admission, then the total teacher cost will 
increase to $5,000 (2 times $2,500) and the new unit cost will rise to $98.04 ($2,500 divided 
by 51) as indicated at point B. Cost will once again begin to decline as new students are 
added. Eventually, when there are 100 students, the unit cost (at point C) once again will 
be $50 ($5,000 divided by 100) or the same as at point A. 

This relationship is an importanlone in understanding the conceptof economics of scale 
within a class and within a single school and in understanding administrator approaches to 
teacher utilization. One often hears comments about small classes being "uneconomic." 
This is a special case of the general error of confusing economy or efficiency with 
inexpcnsiveness. Small class size does mean higher unit costs; however, as was noted 
earlier for the case of rural schools, such a situation is only uneconomic when the practical 
considerations and production conditions would allow for larger classes. In many ^^ral 
areas, for example, it is not possible to have large class sizes (especially in the upper grades 
where prior attrition has had an effect on the available candidates). Also within the 
production conditions of the teacher-centered classroom there are few ways to alter the unit 
cost with the exception of resorting to multiple class teaching — an alternative that can pose 
problems in terms of achievement levels. 

Another case of justified higher unit cost is where the subject matter (e.g. laboratory 
science) or the nature of the students (e.g., learning impaired pupils) require much greater 
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individual attention than a larger class will allow. In summary, analysts need to be very 
careful about identifying unit cost variations with inefficiency when at best, higher unit 
costsarea possible symptom; at worst, they canbea totally misleading indicator for policy 
formulation. The solution is to have more information on unit costs and more mformauon 
on the classroom context within which the costs are generated. 

Before concluding this discussion of educational costs, a small digression is justified on 
the "perception" of educational costs. It is one of the common charactensucs of 
educational production that the perception of cost varies depending upon the role of the 
individual within the educational hierarchy. The parent or student may view all costs as 
given or fixed with the exception of student time. In a society where chQd labor remains an 
importantcontributing factor to family welfare, the amount of time required for schooling 
andits incidence within the work day and across the calendar year will have a dramatic 
effect on the willingness of parents to release children to participate m school acuviues. 
Depending upon cultural standards, parents may have different opportunity costs (the value 
of the perceived sacrifice in allowing the chQd to attend school) for male versus female 
children- when combined with labor market biases in favor of males, the net. if not 
aggregate, effect of these cost and benefit comparisons is normally in favor of male 
education over female education (with the inevitable result of maintaining across genera- 
tions the very gender inequality that education programs often are designed to ameliorate). 

Mandatory schooling, if enforced, has the effect of reducing the legal opportunity costs 
of child participation in schooling to zero. It does not affect the real sacnfice to the family 
of course and that is why enforcement not pronouncement of compulsory schooling is the 
key determinant to changing family behavior. While mandatory participauon laws or 
regulations may require participation they do not. by themselves, assure regular atten- 
dance, retention, or motivauon in learning. A major problem in many developmg nation 
is that compulsory education laws have been instituted prior to the estabhshment of a school 
system that can benefit most students. The result can be a disillusionment with educauon 
bv some parents and children and an abandonment of the school system for pnvate and 
nonformal alternatives or even for a return to traditional child or young adult forms of 
employment. Forsomesub-Saharan African sociedesthefailureoftheeducauonalsystem 

has been coincident with the failure of general economic development. Thus, the 
opportunity costs of education are lower (because of fewer jobs for children and young 
adults) and some children may be caught between the equally dismal altemauves of a 
seemingly ineffective education and a labor market which requires increasmgly high 
educational credenuals for even the most rudimentary of modem sector iobs. 

While the costs of student participation are quite real to the parent and sUident they often 
are ignored by the teacher in designing instructional conduct in the classroom. Vrewing the 
student time as "free" may lead the teacher to institute acUvities that make poor use of 
suident time, including leaving students to wait for further instructions or assignment 
While some time off-task is inevitable, the danger in the teacher havmg an unconcerned 
attitude toward a proper utilization of student time is that all students, but especially the 
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more advanced students who are likely to complete assignments more quickly, may 
develop negative classroom attitudes and bad work habits. 

As suggested here, the dramatic difference in the student versus teacher judgment as to 
the value of student time is at the heart of many classroom problems. Teachers are vested 
wi>Ji the authority to use both their own time and that of the students; thus, there may be 
excessive use of the lecture format because this approach economizes on teacher time even 
though it may be wasteful of individual student time relative to small group, tutorial, or self- 
study alternatives. A proper administrative approach would require that the teacher's 
choice of instructional technology consider the cost of all inputs and not just that of their 
own time and effort 

The complaint of the student vis-d-vis the teacher often is echoed by teacher complaints 
about parents, supervisors, and administrators. Many teachers may feel that the "manag- 
ers" of the schools are making management decisions as if there were a zero marginal cost 
to using teacher time. Thus, decisions are made to change the educational process by 
increasing the demands on teachers rather than by supplying the teachers with the 
complementary inputs necessary to make the teachers more effective. The rationale is that 
teacher responsibilities can be increased without affecting the nominal school budget 
which is not the case if additional instructional support materials or equipment are 
provided. What often results, of course, is that the new responsibilities are either unfulfilled 
or exert a real "cost" in terms of general teacher motivation and effort (and can lead to 
reduced retention of teachers and substantial concomitant costs for the training of new 
teachers and the loss of expertise). Given the salary of teaching in most developing 
countries, anything which increases the burden to teachers of remaining in their occupation 
will drive out of teaching those who have the best opportunities in other employment. 
While there is unlikely to be a perfect correlation between the pedagogical ability of 
teachers and other job opportunities, the correlation is assumed to be positive: the 
implication is that there will always be a tendency to lose the better teachers first if the 
"cost" of being a teacher is increased substantially. 

Finally, individual school administrators often feel thatcentral government bureaucrats 
or politicians may not consider the implications for local school costs in the pronouncement 
of new programs or policies. There is no more consistent pattern of policy practice in 
education over the last twenty years, in both developed and developing nations, than for 
central authorities to devolve new responsibilities on the local school without a concomi- 
tant increase in the resources with which to meet these new responsibilities. The foregoing 
is not a specific criticism of attempts to increase the level of effort of educational 
practitioners; rather, it is a general warning that a failure to consider costs — in all their 
forms — may lead not just to the failure of new reforms but to the creation of counterpro- 
ductive results. 



* Support from non-economists for the important role of cost analysis can be found in 
Friend (1985), Cummings (1986), and Postlethwaite (1987). 
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From this brief discussion it may be seen that there is policy insight to be gained from 
the analysis of cost issues even when effectiveness measures are not available.* Whatever 
the contribution of cost analyses alone, however, their real value lies with their use in 
conjunction with the effectiveness measures discussed earlier in this report. In the next 
section a listing and brief discussion will be presented dealing with the alternative 
indicators of educational efficiency that may be produced when one has both cost and 
effectiveness data. 

n. ALTERNATIVE EDUCATIONAL EFFICIENCY INDICATORS 

The following discussion of efficiency may appear almost anticlimactic given the 
detailed discussions of effectiveness measures. As noted above, any valid effectiveness 
measure, when combined with cost data, can be used to indicate the degree of efficiency 
with respect to the resource measured. However, in the discussion that follows the 
emphasis will be on those effectiveness measures— such as number of students, graduates, 
achievement, or earnings— that are most frequently used to indicate educational efficiency. 

The discussion will be organized around the four models of efficiency analysis presented 
earlier: benefit/cost, cost-effectiveness, least-cost, and cost-utility. Within each category 
the discussion will focus on examples of the effectiveness measures appropriate to the 
particular model as well as some further methodological issues that arise relative to the 
specific model or its operationalization. The discussion is not exhaustive and does not 
include all of the aforementioned effectiveness measures. As was noted in discussing them 
in detail, some measures simply are too narrow to be used alone as a general efficiency 
indicator. Others simply cannot be operationalized at a reasonable level of objectivity and/ 
or for a reasonable data expense. However, the Ust of indicators presented here are 
important in themselves and as prototypes for other indicators that may be developed if 
alternative and preferred measures of effectiveness become available. In summary, the 
indicators discussed below are indicative of the major range of indicators one is likely to 
encounter in educational policy analysis in either a developed or developing nation. 

A. Benefit/Cost Analysis 

In business situations the direct outputs of the production process have a financial value 
(based upon market or social judgment) that is stated in monetary terms. Thus, the 
comparison of benefit/cost ratios for business alternatives is acommon means of promoting 
rationality in the decision process. Whether the benefits and costs are for a single time 
period or occur over multiple time periods has no effect on the validity of the benefit/cost 
criterion: it is equally suitable for either consumption or investment decisions. 

Education and the educational production process are not directly analogous to the 
situation in the business sector. The direct outputs of education, such as attainment and 
achievement results, are not directly expressible in financial terms. To identify a benefit 
of educatic^n in such terms requires shifting to the less direct outcomes such as employment 
and earnings. Employment effects themselves are interpretable in financial terms only to 
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the extent they affect the probability of receiving different patterns of future earnings or of 
reducing obligations for social support such as unemploynficnt and welfare transfer 
payments. Also, the consumption aspects of education are rarely considered directly; more 
often, they are treated as a residual effect or as an explanation for expenditures on education 
in excess of what can be justified by the investment criteria. 

Because education takes place over time and its results (especially those related lo 
earnings) occur over ar. even more extended time, two special forms of bcnefii/cost analysis 
have been used in studying educational investments: the present value of benefit/cost 
approach and the rate of return approach. Both models are based on net benefit and cost 
relationships such as those shown in Figure Four, Parts A and B. In Part A, there is a single 
net cost period followed by aperiod of varying net benefits. This is analogous to the normal 
understanding of a period of education and training (during which direct cx)sts and 
opportunity costs are incurred) followed by a period of higher earnings. Part B indicates 
a pattern of recurrent net cost as would occur if an individual had to interrupt employment 
periodically for new or refresher training. 

It is critical to understand that the diagrams represent net costs and benefits to the 
individual for every single time period. Benefits from education may occur while the 
person is still in training and the person may have additional costs during the employment 
period to maintain the value of their education. 

The concept of opportunity cost was raised in the original discussion of efficiency and 
educational production. If training or education requires that an individual sacrifice 
employment or leisure time then the value of this time is a "cost" of education. Most 
commonly this is operational izcd in terms of the foregone earnings of the individual — the 
reduction in earnings as a result of the time spent as a trainee or student. Similarly, the 
earnings benefits must be not total earnings but the increase in earnings as a result of the 
education or training program. 

The nctbcncfilcur/es presented in Figure Four are for individuals and would be relevant 
for individual decisions about education. If one were analyzing social decisions about 
educational investments additional factors would have to be considered since some costs 
and benefits would not appear a? a direct effect upon the individual student or trainee. For 
example, the amount of subsidy (of tuition, housing, food, etc.) paid by government or other 
social agencies to assist the student/trainee should be added as a cost factor to the amounts 
paid directly by the individuals; similarly, some authors insist that the higher amount of 
taxes paid o*^ higher incomes by graduates is a benefit to society and should be added to the 
net amounts received by the individual (Windham, 1981 , suggests that only the difference 
between the educated person 's increased tax payments and his or her increased use of public 
services should be so considered). Externalities, both positive and negative, also must be 
included in the analysis in order to facilitate the social decisions and to increase their 
rationality. 
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FIGURE FOUR 

INCIDENCE OF NET BENEFITS OVER TIME 

A. SINGLE NET-COST (INVESTMENT) PERIOD 




Tune 



C-) 

B. MULTIPLE NET-COST (INVESTMENT) PERIODS 
N«t Benefits 



(-) 



Time 



lie 
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Given ihc data in Figure Four (understandixig that it may be private or social, for an 
individual or for a group), it is possible to calculate either a present value of beneVii/cost 
ratio or a rate of return siatisUc, The details of present value analysis can be found in any 
undergraduate finance text The standard reference [or rate of rcium analysis is Psacharopoulos' 
Returns to Education (1973). However much one may disagree with his application of the 
methodology (Klecs, 1986). no better treatment has been provided of the basic concepts of 
rate of return estimation. 

Both forms of analysis depend on comparing benefits and costs over time. The present 
value appr-ach calculates the time discounted value of benefits minus costs for all the ume 
periods in which benefits or costs occur. The formula used is: 

n 

pv = Z . 

(I + r) » 

t = l 

the sum of the present values of the net benefits (Bt-Ct) 
benefits in each time period 
costs in each time period 
number of time periods "t" 
rate of discount. 

For an educational investment to be justified the present value of net benefits must not 
be negative (if they are zero the investment leaves the investor's present condition 
unchanged) and the present value must be at least equal to that of alternative investments. 

While the present value approach is logical and obvious and has pedagogical merit in 
training individuals in the investment concept, it has not been the more popular criterion. 
Rather, the rate of return approach for education has dominated the economic literature in 
the same manner that rate of return has been the preferred criterion in business investment 
analysis. One would use the same formula as above but, rather than using an externally 
defined rate of discount (most commonly the cost of borrowing funds), one solves the 
equation to find the unique rate of interest that will cause the present value of the sum of 
net benefits to equal zero. By definition, this is the same rate of interest that sets the present 
value of benefits equal to the present value of costs; it thus establishes the rate of interest 
at which the project will "break-even," 

This approach satisfies the criterion of non-negativity, since if Uie calculated rate of 
return exceeds the appropriate external rate (the rate of borrowing, for example), one can 
be sure the present value of net benefits is positive. However, the rate of return approach 
should be avoided in comparing mutually exclusive alternatives (forms of education or 
educational/work alternatives that cannot be pursued simultaneously). Ray (1984) notes: 



where: PV 

c. 

n 
r 
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In such comparisons, the project with the highest rate of return is not 
necessarily the one with the higher NPV and is therefore not necessarily 
the best project. While a variant of the rate of return technique can be 
used to indicate the correct choice in such cases, it is usually cumbersome 
and prone to error. The rate of return technique is therefore not fully 
satisfactory. 

A (cynical) explanation for the continued use of rate of return analysis in spite of this 
methodological weakness may lie in its apparent complexity to noncconomists. 

This debate aside, the investment analysis criteria as indicators of educational efficiency 
will and should continue to be used. Those technicians who misuse such analyses are no 
more culpable than are the administrators and others who have not made the effort to master 
the concepts and thus are not able to interpret the results of such analyses for themselves. 
As with all efficiency analyses, the investment criterion of benefit/cost is not the sole 
answer but can be a significant contributor to better understanding and thus to better 
answers. 

The educational system, and the economic environment in which its operates, will need 
better answers desperately in the next two decades. Ignoring the effect of education on 
earnings is foolish both professionally and politically for policy analysis; failing to improve 
the quality of the analysis of this relationship through proper use c fbcnefit-cost approaches 
would be equally foolish. 

B. Cost-Effectiveness Analysis 

In moving from financial benefits to quantitative effects one greaUy expands the number 
of output and outcome measures of effectiveness that may be combined with cost to 
generate an indicator of educational efficiency. Of the vast number of alternative forms of 
indicators that could be discussed based on the earlier survey of educational effectiveness 
measures, five indicators will be emphasized here because of tiieir frequency of use. 
general availability from standard data sources, and meaningfulnc ;s for policy analysis: 

- unit cost 

- cycle costs 

- atuition cost 

- cost per unit of achievement 

- cost per unit of dispersion 

Unit costs may appear an unsuitable indicator of educational efficiency since it meas- 
ures only total costs divided by the number of studc'\ts (or tiie total of an ingredients-based 
summation of the various cost inputs for an average student). The purpose of unit cost 
calculations, however, is to allow one to compare tiie available, even if minimal, 
quantiiative'daia on e4ucation with qualitative information and inferential analysis to 
identify areas of potential inefficiency. This Uien allows Uie analyst to study Oie specific 
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problems and opportunities that exist for improving effectiveness (as equated with size of 
enrollments) for a particular level arid type of education. Although not sophisticated, unit 
cost analysis is often all that the availability and quality of cost and effectiveness data will 
permit 



TABLE ELEVEN 

PUBLIC PRIMARY SCHOOLING IN BOTSWANA: 
ESTIMATES OF UNIT COSTS AND CYCLE COSTS 1983/84 

L UNIT COSTS 

Cost to Parents P 15 

(uniforms & misc.) 

Cost to Govemmenl 

MOE P136 

MLGL 18 

Local 20 

Subtotal 174 

Total Cost P189 

IL CYCLE COSTS 

Year Pupils per Standard* 

One Two Three Four Five Six Seven 

One 1,000 

Two 978 

Three 969 

Four K037 

Five 882 

Six 833 

Seven 1,083 

A. Cost of an A or B pass = 21 .4 years or P4»045 

B. Cost of an A.B.C pass = 10.3 years or Pl,947 

* Assumption of progression rates based on Ministry of Finance and Development Planning 
projections for 1983 forward. 

SOURCE: lEES Project, Botswana Education and Human Resources Sector Assess- 
ment, 1984. 



110 



110 



The Analysis of Indicators 



Where enrollment data allow, it also is recommended that cycle costs be calculated. 
Cycle cost is defined as the average number of student years of education provided by the 
educational system relative to every graduate produced. It is calculated from a table of past 
enroUment patterns or a table of enrollment projections. Cycle cost is not the average time 
it takes each graduate to complete the cycle. The cycle cost indicator includes the years 
of education (including repetition) of the graduates plus the years of education (again 
including repetition) of all non-graduates. 

A comparison of financial unit and cycle costs (years multipUed by the cost per year) is 
presented in Table Eleven (from the Botswana sector assessment of 1984). The unit cost 
data has been derived for 1983/84 by the inputs (ingredients) approach and is equal to Pula 
189 per year. Part II of Table Eleven indicates the expected progression rates mcluding 
repetition for the seven year primary cycle that existed in Botswana in 1983/84. For every 
1 000 students who begin standard (grade) one in Year One. the table depicts how many 
students will be in each succeeding grade and year. The high figures for standards four 
and seven are because of the higher rates of repetition in those two years. 

Out of every 1 .000 students who begin. 3 17 are expected eventually to receive an A or 
B pass and 658 an A.B. or C pass on the national primary school leaving examination. Thus 
the cycle cost for this example is equal to the number of passes on the examination 
(graduates) divided by the total student years of education (6.782 = 1.000 + 978 + 969 + 
1 037 + 882 + 883 + 1.083). Thus, the cycle cost for A and B passes is 317/6.782 or 21.4 
student years; 21 .4 years times the unit cost of Pula 189 results in a financial cycle cost of 
Pula 4 045 If one uses the more generous definition of A.B. and C passes to define 
graduates, then the cycle cost in years is 10.3 (658/6.782) and in financial terms is Pula 
1 947 (10.3 xP189). Again note that no graduate is expected to take 10.3 and certainly not 
21 4 years of education to finish the seven year cycle; and yet. these years of education will 
have to be provided by the education system because of the effect of repetition and attrition. 

A possible weakness in the cycle cost methodology is that it values only graduates. 
Where school leavers prior to graduation are determined to have derived significant benefit 
from schooling some adjustment can be introduced to grant partial value to such school 
leavers. The most common example of this is in primary education where the achievement 
of literacy and numeracy will, even if the individual does not graduate from the pnmary 
cycle, have significant social and personal effects. A counter argument to tiiis concern is 
the fact that labor markets, at least at the entry level, are keyed to graduation levels and 
certificates and not to years of attainment or acquired skills per 5e. One need only compare 
the earnings or employment of graduates and near-graduates of secondary school and 
higher education institutions to sec the impact of the labor market fixation on graduate 
certificates. 

A third and related efficiency indicator that can be calculated from basic data is the 
attrition cost index (in part, a misnomer because it includes both attrition and repcution 
effects) It is based upon the ratio of cycle cost to the product of unit cost multiplied by the 
number of years in the schooling or training cycle. For example, for the Botswana datacilcd 
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above, ihc cost per primary cycle graduate ideally would be Pula 1 ,323; this is the product 
of unit cost of Pula 189 times the seven years required in primary education. When this 
value is compared with the actual cycle costs, one derives an attrition cost index of 3.06 
(4,045/1,323) for A and B level graduates and 1.47 (1947/1,323) for A,B, and C level 
graduates. It should be recognized that since the unit cost figure is in both the numerator 
and the denominator, cycle and attrition costs may be calculated from student years alone. 
This is important to remember if unit costs are unavailable or unreliable and one still needs 
an efficiency indicator incorporating attrition, repetition, and graduation rates. 

Unit and cycle costs and the attrition cost index operate under the assumption that 
students or graduates are the desired output of education and that enrollments and 
graduation rates are acceptable effectiveness indicators. The enrollments would be 
completely valid as an indicator only if graduation had no pay off, per se. The graduation 
measure is an adequate indicator only if one can be suje what the status of "graduate" 
implies in terms of the 'cognitive and noncognitive attributes valued by the market or 
society. Because neither of these assumptions is fulfilled in reality, one can use these 
efficiency indicators only with great care and in full knowledge of their narrow conceptual 
base. 

To widen that conceptual base, researchers have tried to measure the specific achieve- 
ment of students and graduates. The methodological and interpretive limitations on tliesc 
measurements were discussed earlier. Assuming these can be eliminated or, more 
probably, controlled for in the analysis, the comparison of costs with effectiveness in 
achievement would be an improved indicator of educational efficiency. What the analyst 
seeks to identify is whether a change in expenditure can lead to a change in achievement. 

Of course, the goal of efficiency analysis is to identify the most efficient not the most 
expensive forms of education. The researcher must be assured or prepared to assume that 
the additional expenditures are allocated across inputs in such a manner (i.e., within the 
most effective technology) that the expenditures are focused on the inputs that can make 
the greatest contribution to the desired output or outcome measure(s). The desired measure 
or measures of outputs or outer mcs must be scleeicd based on the value judgments of the 
key decisionmakers. For example, if the key decisionmakers are interested in the effect of 
education on national economic development it would not be useful to supply them with 
information on the efficiency with which education promotes enhanced appreciation of art 
(unless one can show a causal or coincident relationship of art appreciation with economic 
development outcomes). Similarly, if the key decisionmakers primarily are interested in 
education's effect on student and graduate political opinions and loyalties, they may find 
little value in information on the average effect of education in terms of enhanced 
mathematical skills. 

The achievement measure chosen must be selected based upon the preferences of the 
users of the efficiency information. The most common subject areas selected are language 
and mathematics scores because of the centrality of these two topics within the school 
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curriculum. Achievemenlin social studies, civics, art, science, etc., potentially are equally 
valid but they have been less commonly used measures of academic achievement 

When multiple subject areas are of interest, it is preferable to analyze them individually 
rather than to create an artificial index (by creating a weighted or unweighted average of 
the achievement scores across rubject areas). The meaning and relevance of an index or 
average such as this is uncertain and will not be as readily interpreiable by decisionmakers 
as will the separate results by subject area. An additional advantage of presenting the 
individual scores to decisionmakers is that it forces them to weight the individual values 
(language versus maUiematics skills for example) and prohibits the researchers from 
imposing their own values as to the relative importance of the subject areas in an implicit 
manner that may not be recognized by the decisionmakers. 

Once the achievement effectiveness measure is selected (and one has controlled for 
other determinants)* it is "simply" a matter of comparing how the cost variations among 
the sample, or population of cases, affects the achievement measure. In an experimental 
setting both the control of other variables and the isolation of expenditure on the most 
product! ve inputs arc easier to achieve. In what is termed "natural experiments," using data 
from the normal population of education, these controls are exerted statistically. However, 
one can only control for those variables which do, in fact, vary across the population. 

For example, one can test the effect of class size changes on costs and achievement in 
an experimental population by creating classes that vary in si7e but are standardized (to the 
extent feasible) in every other manner of determinant (Cal;j:::/-:tal., 1983, isoneof the few 
cases of an experiment which involved change of class size during the experiment). But 
in the actual education population one may find either that class size varies only within a 
narrow range (because of regulations, teacher assignment policies, or some other cultural 
or bureaucratic standard) or that it varies outside this range only in cases correlated with 
other determinants such as size of place or multiclass teaching practices. An example is that 
small upper-primary classes in many countries exist but they are so highly correlated with 
rural location that statistically one cannot separate the causal effects. Similarly, in some 
countries, large classes may be identified with urban areas where population is greater or 
with rural areas where teacher supply is more of a problem. The point to be remembered 
is that statistical controls are not always fully effective in the analysis of non-experimental 
data. 

The issue of class size (mentional abc^^e and in the eariier effectiveness section of this 
report) illustrates this point. Many studies in the U.S. and Europe in the 1960s and 1970s 
found no class size effects on measured achievement. This is hardly surprising given that 
class size variation was concentrated in the range of 25 to 35 students. However, one can 
conceive of class size effects without expecting them to have an impact within the narrow 

* To compare costs with achievement outcomes wL^iout controlling for other determi- 
nants is methodologically unsound and requires c xtreme caution in the interpretation of 
the results. 
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range thai exists in the standard school or training center. Logically, a tutorial (one student 
to one teacher) would be a superior form of instruction in certain settings, especially if 
cognitive achievement is the primarily desired output. More importantly, the monolithic 
attitude of donors and national administrators concerning the irrelevance of class size has 
restricted the responsiveness of educational systems to the special needs for remediation 
activities for physically or learning disabled students and the special requirements of 
certain courses of study such as laboratory sciences and foreign language (see, for example, 
the recommendations for remote rural Chinese schools in the China Study Report, 1986). 

The le§son to be learned from the cost-effectiveness studies of the last three decades is 
one of caution in interpretation and the need for greater care in the conduct of such research. 
A final example of the latter is that cost effects are almost always more easily attributable 
to an instructional change or innovation than are achievement effects. The danger is that 
an inherently conservative if not negative attitude toward educational changes (and 
certainly experimentation) can develop. The solution is not to offset this by manifestation 
of an evangelical zeal on the part of the change advocates. Rather, the decisionmakers 
themselves must become more competent in questioning research design and conduct and 
in interpreting research results. With greater competence will come greater confidence in 
their ability to monitor educational change and reform. 

The analysis of cost per unit of achievement always must assume that dispersion 
(variation in individual achievement results) is constant or irrelevant. However, given the 
importance assigned by most societies to education as an equalizing force, it is possible to 
design an efficiency study that would look at cost per unit of dispersion as one indicator 
of efficiency (with controls to assure that reduced dispersion is not concomitant with 
reduced average performance). Any measure of dispersion could be used but the standard 
deviation and Gini coefficients (see Appendix) are the most useful measures. Again, in line 
witli economic theory, one is interested in the effect of a change in cost on the change in 
the measure of dispersion. 

The instructional technology as well as the relative and absolute use of inputs must be 
considered in determining effects upon the equality of achievement results. The methodo- 
logical issues of control and determinacy are exactly the same as discussed above for cost- 
effectiveness analyses that use mean achievement as the effectiveness measure. The 
infrequencyof cost-effectiveness studies that focus solely on dispersion indicates that both 
policy analysts and, one assumes, decisionmakers consider equality of outcomes a 
secondary, albeit legitimate, efffxtiveness measure. 

In fact, one of the great needs in educational policy analysis is to advance the position 
of dispersion of results to parity with that of mean achievement. Concern about 
achievement mus( be concem about the distribution of achievement, not just its central 
tendency. Not only is every child of equal importance — although some studies either 
ignore or are oblivious to this point. The outcome effects of education are a function of the 
distribution or mix of output characteristics and not just a function of the mean level. The 
labor market responds to the differences in workers as effectively as it does to thein common 
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traits; to date most educational research has not been as attuned to measures of dispersion 
as they have been to measures of central tendency and this weakness needs to be remedied. 

To many economists and educational researchers, achievement equality is not a subor- 
dinate goal but must be concomitant to changes in achievement means. One can make a 
policy decision to sacrifice equality in order to promote mean achievement or even to 
sacrifice the achievement results of part of a population to benefit another part of the 
population. But this should be a decision based on sound knowledge of the relationships 
between educational costs and both individual achievement levels and differences. 
Otherwise, one may make sacrifices that are unnecessary or institute policies that are 
inefTicicnt in their tradeoff of achievement growth versus achievement equality. To have 
the prerequisite information to make Uiese critical choices, b.^ analyst must possess cost- 
effectiveness data related botii to tiie unit of achievement and to tiie unit of dispersion. 



C. Least-Cost Analysis 

As defined earlier, least-cost analysis seeks to identify tiie least expensive means of 
producing a given effectiveness with the effectiveness measure specified in any form or 
combination of tiie forms discussed earlier. However, tiiis approach does not provide tiie 
analyst license to ignore effectiveness issues. A review of donor and government project 
proposals could lead one to tiiis misinterpretation. While one cannot always prove tiiat tiie 
alternatives under consideration are equally effective, some evidence should be provided 
that Uie differences in effectiveness are of a scale tiiat is irrelevant for tiie current policy 
considerations. If one can only assume equal effectiveness among tiie alternatives, tiie 
analysis might be better described as a form of cost analysis (such as was described for tiie 
Liberian lEL project) and not as cost-effectiveness analysis. 

Least cost analysis can be used for each of tiie four types of effectiveness measures 
discussed earlier: inputs, processes, outputs, and outcomes. Examples of least-cost 
analysis based on inputs would be those tiiat focus on cost per teacher, per textbook, per 
class, or per school. Here, one is defining tiie effect as an input type and seeking to idenufy 
the most efficient way of delivering tiiat input. Similarly, one can compare classroom or 
school costs under varying instructional systems (technologies) so as to assess tiie effect of 
alternative processes on costs (per student or per graduate). The assumption in botii tiie 
input and process definitions of effectiveness must be \hi t tiiese measures are "acceptable- 
proxies for tiie more obviously relevant output and outcome measures of effectiveness. 

Least-cost analysis of output measures is preferable for tiiis reason. Common output 
measures in least cost analysis are attainment rates, achievement levels, and attitudinal or 
behavioral measures. These studies differ from cost- effectiveness in that, in least-cost 
analysis tiie analyst must show tiiat tiie effectiveness measure does not vary or vary 
significanUy rather tiian, as in cost-effectiveness an^dysis, study how tiie effecuveness 
measure varies for a fixed change in cost. Least-cost analysis emphasizes cost differences 
while cost-effectiveness analysis emphasizes tiie changes in botii cost and effects. 
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Finally, outcome measures such as employment, earnings, social altitudes, etc, may be 
used in least-cost analysis although it becomes more difficult to assert that such effects are 
invariable across educational alternatives. The most common form of least-cost analysis 
then is the one dealing with outputs, and within the output category, the most common 
effectiveness measure is that of achievement 

D. Cost-Utility Analysis 

Little additional detail on cost-utility analysis can be presented here except to empha- 
size again the distinctions from least-cost and cost-effectiveness analysis. Cost-utility 
analysis can be based on data from any of the preceding forms of efficiency analysis. It 
differs from the least-cost analysis in that, in cost-utility analysis, the value of both costs 
and effects may be subjectively determined and there is no need to standardize (through 
proof or assumption) the effectiveness side of the equation. Cost-utility analysis also does 
not depend solely on objectively quantified costs and effects of the types found in cost- 
effectiveness and benefit/cost analysis. 

To help clarify these distinctions further, researchers engage in benefit/cost, cost-effec- 
tiveness, and least-cost analyses but decisionmakers (especially administrators and politi- 
cians but including individual parents and students) always engage in cost-utility analysis. 
Rarely can the "objective" forms of efficiency analysis be sufficiently comprehensive that 
one would base educational decisions solely on the ratios or coefficients they generate. It 
is the responsibility of decisionmakers, public or private, to be informed of the quantitative 
findings but the final decisions almost always will come down to a question of applying 
their own values and experiences in interpreting the available data. 

When researchers bewail the indifference shown toward their results by apparently 
uninformed decisionmakers, the researchers sometimes have justification: in organizations 
that have a history of inaccurate, incomplete, and untimely data, a bureaucratic culture of 
personal culpability, and a highly politicized public sector, many decisionmakers have not 
been encouraged to acquire the skills that would allow them to utilize educational policy 
data purposefully. More often, however, the researchers arc failing to appreciate the more 
complex utility determinants of choice faced by the educational decisionmakers. Even if 
the decisionmakers accept the analysts' data, they still must interpret the data for 
themselves in terms of the larger social or political systems within which they operate and 
their own values. 

For example, a common finding in the last decade has been that most developing nations 
have a relative over-investment in higher education and an under-investment at lower 
levels (such findings only rarely deal with the inefficiency — misinvestmcnt — at all levels). 
Researchers often express dismay that such findings do notspur administrative reforms and 
reallocation of resources from higher education to other human resource subscctors. The 
role of higher education, however, is such that it is not subject to reform based on these 
narrowly-defined efficiency indicators. Rather, because higher education serves multiple 
roles in addition to promoting national economic development (e.g., a national or regional 
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status symbol, a service institution for the elite, and an escape valve for the pressures of 
unemployment from an excessively expanded secondary education sector), the narrow 
definition of most higher education efficiency indicators allow them only limited applica- 
bility to the decision process. The decisionmakers do not necessarily ignore the results of 
the efficiency researchers; they may, however, assign the results less weight than the 
researcher would. 

Convergence between research results and decisionmakers' premises can only occur 
through the increased use of multiple indicators of efficiency. As noted in the inu'oduction 
to the economic concepts related to efficiency, the definition of efficiency is a function of 
the definition of educational goals — the desired outputs and outcomes. Since it is possible 
for a single individual (stakeholder) to have multiple goals,and since almostall educational 
activities have multiple stakeholders, the use of single dimensions of effectiveness based 
upon a narrow definition of cognitive achievement or financial success is dramatically 
inappropriate. 

Given the serious methodological limitations faced by all educational analysts and the 
special problems of applying economic concepts to such a complex activity as education, 
the solution will not be found only in improved technical devices for measurement or even 
by more expensive analyses. Objective data can be improved but even the best objective 
data will not eliminate the need for subjective judgments by decisionmakers. In the final 
instance, all educational decisions — from the individual student-teacher interaction to the 
formulation of national policy — are cost-utility decisions. 

The researchers' responsibility is to widen the efficiency definition to include more 
outputs and outcomes and to improve the accuracy , breadth, and timeliness with which this 
data is provided. The most dramatic example of what happens when researchers fail to do 
this is the sad history of educational innovation projects. Whether the innovation is 
educational radio or television, programmed teaching or instructional materials, or any 
other attempt to affect traditional classroom practice, the myopic focus of the researchers 
on achievement results alone has been a primary reason why dissemination of these 
innovations has been so rare and so slow. Achievement results are important, but so are 
costs, so are administrative changes, so are parent, teacher, and public attitudes. The 
undimensional definition of educational effects has led the educational innovations to be 
experimental successes (by their own narrow definitions) but dissemination failures. 

In some cases these failures are good things; administrators or others may have recog- 
nized what the researchers did not: the innovative system would not have operated 
efficiently outside the **grccnhouses" of the controlled innovative classrooms and schools. 
Unfortunately, many of the*Tailed" innovations would have been of substantial benefit to 
many children and, eventually, to the larger society. In these cases, the failure of the 
researchers becomes a failure for the educational system and for the society. 

The culpability of the researchers lies in their unwillingness to identify the appropriate 
efficiency indicators before the researchers begin their work. Instead of assigning an 
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efficiency indicator the researchers feel is important (and often selected because of its 
relative ease of measuiement), the researchers should have engaged in identifying the 
cridcai stakeholders and eliciting from them the appropriate measures of efficiency. Not 
ail of these will be easily operationalized and, for some, the cost-effectiveness of their 
collection will not justify including the efficiency standard in the research. But identifying 
the multiple indicators that stakeholders feel are important will improve the relevance of 
the quantitative results and alert the researchers to the data gaps in the policy relevance of 
their work. Knowledge of the latter can help researchers prepare the presentation of their 
results in a manner such that all stakeholders will understand better why certain outcomes 
or outputs important to them are not part of the research results. 

If the above discourse sounds uncomfortably close to a description of needs assessments 
or marketing surveys, that should not be surprising. The concept of "social marketing" that 
has had such a salutary impact in health and agriculture dissemination activities has not yet 
been utilized effectively by educationalists. Social marketing has two main initial 
functions: to identify the wants and needs of the subject population and to promote new or 
altered definitions of individual wants. Applied to efficiency analysis, the first function is 
fulfilled by identifying the outputs and outcomes the various stakeholders to education 
believe are important. The second function is more proactive in that one attempts to 
introduce new output/outcome goals or to alter existing ones. 

For example, parents may desire economic success for their child but not understand 
why mathematics skills are a relevant indicator; the researcher cum social marketer for ef- 
ficiency analysis can attempt to show parents how certain educational skills can promote 
the goal the parents already have (thus altering the parents* understanding of effectiveness 
or efficiency analysis based on mathematics achievement). In another case, teachers or 
administrators may not see cost containment as an issue relevant for them. By convincing 
them of the alternative uses of time and resources in the classroom and of the dire 
consequences that will follow from the exhaustion of national fiscal capacity, a new 
standard of efficiency incorporating cost considerations may be accepted by these stake- 
holders. 

The discussion of efficiency indicators completes the review of the application of the 
economic concepts' of production and utility to educatioi . Before proceeding to the 
summary discussion and the presentation of recommendations for research and policy, the 
next section will present a discussion of the role of efficiency analysis in the creation and 
maintenance of educational management information systems. This discussion is included 
here because of the importance of institutionalizing efficiency analysis within the normal 
workings of the education system. To date, efficiency analysis has been an ad hoc 
occurrence in the management of educational institutions and systems; since the major 
impact of efficiency analysis can best be realized from the cumulative impact of its 
recurrent use, the present situation is one that, if maintained, will continue to limit the value 
of efficiency analysis and retard the efficiency of educational operations. 
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EFFICIENCY ANALYSIS AND EDUCATIONAL MANAGEMENT 
INFORMATION SYSTEMS* 



The relationship of efficiency analysis to the creation and use of an educational manage- 
ment information system (EMIS) often has been misunderstood. Efficiency analysis is not 
a means of using the EMIS, it should be the means of designing the EMIS. Efficiency 
analysis does not say just what can be done with data but, more importantly, establishes 
criteriafordetermining whatdatashouldbecollected. This latter contribution is especially 
important in that the present EMIS operations in most developing nations suggest that 
tradition and ease of collection often are primary criteria used in the identification of data 
for collection. 

Why should efficiency, rather than quality or equity, be the organizing principle for an 
rsMIS system? Quite simply the efficiency concept incorporates the most inclusive set of 
criteria one could have for assessment or evaluation of an educational system or of its 
components. The efficiency concept is inclusive of concerns for quality or equity , whether 
these latter concepts are defined in terms of inputs, processes, outputs, or outcomes. In 
addition, by giving equal place in the analysis to both costs and effects, the efficiency 
concept is more responsive to economic realities and more responsible in terms of 
recognizing the legitimacy of other social and individual uses of resources. Finally , as was 
suggested in the efficiency chapter, there is a direct link between understanding how to use 
efficiency data and conceptualizing the design of an EMIS in terms of multiple indicators 
and multiple stakeholders. 

Because management information systems (MIS) have been developed primarily by 
non-economists, there has not been the emphasis on a central organizing principle for the 
systems that one might have expected given that MIS originated in the systems analysis 
work of Simon (1977). Simon's basic structure of systems analysis parallels that of 
efficiency analysis in that one begins with problem definition and proceeds through 
establishment of criteria to the proposal and evaluation of alternative solutions to the 
selection of an "optimal" choice. This is exactly the economic model of choice and was 
adapted by— rather than originating with— Simon from classical as well as neo-classical 
economic literature. Efficiency analysis is, in fact, an application of systems analysis 
where one seeks to optimize the interaction of costs and effects within constraints on 
available resources including information. 



* The discussion presented in this section has benefited from the review of the EMIS- 
rclatcd literature in the lEES Project's Issues and Opportunities for Energizing Educa- 
tional Systems (1987). Some of the current presentation is directly traceable to that 
excellent summary prepared primarily by Jerry Messec of The Florida State University. 
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Information as a scarce resource and as a resource subject lo cost-effectiveness consid- 
erations has been a major contribution of the work of Simon and of his followers in the MIS 
and EMIS fields. Paralleling the developments in information system concepts in the last 
thirty years has been an even more dramatic development in the equipment (hardware) by 
which information can be processed. Krocber and Watson ( 1 984) note the dangers inherent 
in the fascination of planners with the high-technology hardware of MIS (as opposed to the 
poor quality of data sources and decision-criteria which have not kept pace with the 
developments in MIS equipment). In stressing what a management information system 
docs rather than how it does it, MIS reformers are attempting to rebalance the information 
field in line with concerns expressed here about efficiency data and their use. 

The simultaneous development of computers (and their heightened sophistication) has 
led to a common confusion that MIS is a computer system. In fact, MIS have existed ever 
since the first systematic collection of data and such systematic collection can be traced to 
the earliest records of civilization. The improvement of MIS requires two major changes: 
(1) the ability to identify data needs of users and cost-effective means for the collection of 
this data at a level of acceptable quality, and (2) more timely and detailed presentation of 
data in a form readily intcrprclable by the users. Computers have helped in the first instance 
by facilitating certain forms of collection and, more importantly, by reducing some forms 
of transcription and aggregation errors common to pre-computer systems. In the second 
instance, computers have certainly reduced processing time for large data sets and have 
allowed much greater detail in the presentation of results. 

Given the significant contribution of computers, there are still two important steps left 
if a sophisticated and n -ponsive MIS or EMIS operation is to be established: (1) the 
formulation of better criteria for data collection and articulation and (2) better training for 
data users so they can make use of the data that will become available in greater quantity 
and detail. These steps cannot be achieved through a further emphasis on hardware 
development. Both require a new emphasis in terms of the methodological approach taken 
to the role of information in decisionmaking. Again, the conclusion of this report is that 
efficiency analysis and its subsumed body of concepts provide the best organizing 
principles both for the establishment of data criteria and the training of data users. 

L DATA CRITERIA 

The critical task in designing an EMIS is the definition of information needs. This can 
be done in one of three main ways. First, information can be collected because it has 
"always" been collected and/or because it is relatively easy to collect (the emphasis on 
enrollment data versus achievement data is explainable in this way). Second, one can 
conducta"felt-need"analysis of major decisionmakers in whichoneasks them to articulate 
the types of information they require and to assign priorities among the information types. 
Third, one can impose on the system a set of criteria based on theory and experience but 
related more to what the MIS professional feels is needed rather than what the end-user or 
decisionmaker feels is needed. The proposed use of efficiency analysis as an organizing 
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principle for an EMIS will involve the integration of both the second and third ways of 
identifying data needs. 

The use of a felt-needs approach alone can. encounter a variety of problems. One 
example is that the decisionmakers may not be able to explain in adequate detail the type 
of information they require. Many organizations fail to express clear decisionmaking 
criteria or, even if they express them, do notapply them in a significant proportion of their 
operations. 

Matthies and Matthies (1977) describe the possible frustration thai may be encountered 
by information specialists who interview managers in an attempt to elicit decisionmaking 
details: "Frustrated MIS designers may accuse managers of not adequately understanding 
their work, while frustrated managers may argue that the designer is not able to comprehend 
their organization." The tension between information designers and users lies in the fact 
that the designers seek to simplify the decision process into its objective and measurable 
components while users operate in a more complex environment where information use is 
influenced both by organizational structures and bureaucratic practices and by cloudy 
criteria for success and a partial and uncertain linkage between decisions and decision 
effects. 

This situation parallels that discussed earlier between the use of objective data to 
establish a framework for the subjective cost-utility judgments of decisionmakers. The 
problem for many information users is their fear that more and better objective data will 
make it increasingly difncult to rationalize (in the non-pcrjorative sense) their inevitably 
subjective decisions. The problem is aggravated by those informauon designers whose 
hubris extends to the point that they resent— and auempt to prevent— any intervention of 
subjectivity in their information system. Such individuals seek to establish mechanistic 
processes based on quantitative data and f ^xed, objective criteria; while some techn ical and 
engineering applications of MIS may justify suchan approach, it is totally inappropriate for 
a social activity such as education. In fact, the intrusion of mechanistic processes may be 
counterproductive in that it can elicit hostility to the EMIS itself by the affected users. 

One must accept the fact that within a complex organization such as an educational 
institution or system one will find decisionmakers who lack the training necessary to do 
their job. One of the most consistent Hndings of the lEES series of sector assessments was 
that the educational bureaucracies were characterized by large numbers of middle-level 
managers who did not have either formal or on-the-job training concomitant with their 
responsibilities. In such a situation it is necessary to develop other means for identifying 
the data needs of the system. Hurtubise (1984) suggests an analysis of the organiz^uon with 
■ the information designers responsible for identifying structure, environment, and the 
planning and control processes (the techniques used would include documentation review, 
observations, and interviews). 

Because of the cariicr-statcd bias of designers toward simpliHcation of processes and 
quantification of data, there is a danger that the designers will develop an inappropriately 
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abstract model of the enterprise. This problem was anticipated by Lucas (1973) who 
proposed the participation of two committees in the design process: 

- A priorities committee would assist the designer in making resource al- 
location decisions and thus avoid conflict between the design of the 
system and its users. The committee would also bring a better compre- 
hension to the design process of why some projects are undertaken, why 
others are denied, and how decisions are made for new activities or 
expansion of existing commitments. 

~ A user committee would includea large representative group of users and 
would involve them in the design process. This would result in better 
informed design and avoidance of possible future conflicts. 

Only through the synergistic efforts of the two committees could it be assured that the 
data criteria of the information system would be both responsive to decisionmakers' 
perceived needs and to the externally determined requirements (based on needs the 
decisionmakers may not be competent in ability or training to perceive or articulate). 

The assertion here is that there may be a third level of needed competence in addition 
to that of organizational decis'.onmakers and information system specialists. There is a 
need for a conceptual framework for decisionmaking that is generic to scarcity and choice, 
not just generic to a single information system or organizational structure. That generic 
conceptual framework is efficiency analysis. Thus, it is asserted here that the principles of 
efficiency analysis are not just an approach to structuring an EMIS but can be viewed 
appropriately as the approach. The efficiency approach defines the alternative types of 
data that can be collected, offers criteria for choosing among them, provides alternative 
decision criteria for using the data with the criteria adaptable to different forms of 
quantification and levels of objective versus subjective valuation, and even suggests the 
types of training needed by data users. No other conceptual approach is so comprehensive 
in the applicability of its parts to educational information and its management as is the 
efficiency concept. And because it may be divided into cost and effects and these two 
concepts are further divisible into subjective and objective values, and the objective values 
can be monetary or nonmonetary, one is presented with a wide range of data specifications 
that may be selected depending on tlic needs of the users. 

II. TRAINING CRITERIA 

The training of data and information users has been a challenge faced by all those who 
desire to improve organizational or system effectiveness but it has posed special problems 
in the education and human resource sectors because of the quantity of managerial or 
administrative personnel, the complexity of the choices they face, and the frequent 
inappropriatcncss of the educationalists' past training. The last is a problem whether 
administrators arc former teachers without training or professional managers without 
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classroom or school administraUve experience. The need for management training is the 
most commonly cited administrative problem in educaUon; in part this is because most 
educational systems promote managers from within the teaching cadre. 

The quesUon with which efficiency analysis can help is: What form of management 
training is likely to be the most useful in preparing educational planners and administrators 
in the use of cost and effectiveness informauon? From the earlier discussions presented 
here, four general categories of training appear necessary for the effecUve educauonal 
manager: 

1. specific skill training; 

2. training in the conceptual framework of efficiency analysis; 

3. training in logic and data-based argument; and 

4. training in the application of skills, concepts, and logic to the 
requirements of their jobs. 

Specific skill training for managers has been dominated in recent years by the attention 
paid to computer training. Too often, the focus has been on training the manager to operate 
a computer rather than on how to use it as part of the informaUon/decision system. It may 
soon be common in the developing world-as it is increasingly in developed nauons— for 
managers to operate their own computer terminals. However, for the present, the pnonty 
need is to develop high-level computer skills among data technicians who can provide 
better data processing for managers. Obviously, basic computer knowledge is valuable for 
managers First, they need to know what data is available and what the data techmcians 
by use of the computer, can do with the data. Second, some of the managerial lack of 
enthusiasm for computers is based upon their concern about subordinate personne'. who 
have skills they, the bureaucratic superiors, do not possess. Basic computer training can 
both allay these concerns and assure more effective coordination between management and 
technical personnel. 

A possibly more mundane but potentially more important set of skills that should be 
improved by management training are the skills of assimilating the informauon rn data 
summaries and reports. All educational managers have experience in this area but they 
often have not had the proper training in how to study a data summary as a means of 
generating further data questions and alternative policy recommendaUons. Similarly, 
technical reports may be impossible for the manager to evaluate with the result that either 
the technicians' conclusions and recommendations may be accepted without proper 
quesuoning of the assumptions or statisucal and personal biases or the report' s potenually 
useful views will be ignored because of the manager's inability to comprehend them. 

To permit managers to process the information in data summaries and reports requires 
the three further forms of training listed above. First, the managers must be trained in the 
conceptual framework of the efficiency analysis. This includes undersunding the 
general framework and definitions and also the ability to comprehend why the efficiency 
analysis is comprehensive and central to management decisionmaking. Managers also 
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need to become familiar with why and how efficiency analysis was developed and the 
specific value and lim itations of its application to education. This training in the conceptual 
fiamework of efficiency analysis should consist of four parts: (1) establishment of basic 
terminology with clear definitions; (2) relating efficiency concepts to the basic terminol- 
ogy; (3) indication of the specific application of educational measures as efficiency 
indicators; and (4) discussion of the statistical, conceptual, and financial limitations of 
efficiency analysis in the practice of actual educational management activities. 

Once familiarity with the conceptual framework has been achieved, educational 
managers need to receive training in logic and data-based argument. To some this may 
seem an unrealistic and unworkable requirement; others may feel it is unnecessary or 
inappropriate because it involves imposition of an arbitrarily selected form of intellectual 
approach on the behavior of the trainees. Training in logic and data-based argument is 
difficult but not impossible. Many programs of study — mathematics, statistics, physical 
science, economics— impose a preferred form of logic on students. Whether described as 
the scientific method, formal logic, or rational behavior, these approaches to reasoning all 
place a premium on questioning of data, testing of alternatives, the relationship of premises 
to conclusions, and the consistency of findings. 

Recently, a joint program of the USAID-financed lEES Project, the Worid Bank's 
Education and Training Division, and the University of Lome was begun with the explicit 
purpose of improving the skills of educational planners in the analyses of data summaries 
and reRorls and the generation of tentative policy recommendations. The experience of this 
activity to date suggests lliat the goal of developing improved reasoning skills is attainable 
but that intensive initial training needs to be supplemented by continuing on-the-job 
reinforcement 

What this project activity has shown, and what experienced educational advisors can 
attest, is that the present skills of educational managers and analysts are underutilized not 
just because of specific skill shortages in technical areas but because of a lack of training 
and experience in data-based argument. The high intellectual skills possessed by many of 
these managers adds to their frustration as they recognize that more can be done to convert 
data into information and to transform educational information into a basis for the reform 
of educational policies and practices. Objective and data-based argument is not arbitrarily 
judged to be a superior means of analysis to anecdotal, personal, and subjective argument. 
It is'deemcd, however, to be a prerequisite to the application of the manager's or other 
decisionmaker's personal and subjective views. Without objectivity, data, and logic, no 
complete and open discussion of present conditions and future alternatives can occur. 

No system of applying logic or intuition can guarantee that "truth" will be discovered; 
however, the approach posed here maximizes the probability of a "correct" decision by 
increasing the basis for discussion and democratizing the access of participants to the 
discussion. The use of data and logic is not a substitute for the experience of managers but 
is a necessary complement in the effort to make educational decisionmaking more 
effective. 
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Finally, educational managers must receive training in the application of skills,coocepts, 
and logic to the requirements of their jobs. This training takes place best on-the-job and 
can consist of ongoing counterpart relationships or of recurrent reviews of decisionmaking. 
In either case the objective of the training is to stress alternatives and justification. What 
are the alternative sources of data used, why were some selected and others not selected, 
and why was the data interpreted in the way it was? The discussion of these points increases 
the managers' sensitivity to the existence of alternative sources, procedures, and conclu- 
sions through demanding that the managers be able to justify their decisions. 

Managers who realize that their decisions must be justified will be more careful and 
deliberate in making decisions. The training process must guard against excessive delays 
caused by concern that decisions will be criticized during review. Two points must be 
established within the organization in this regard. First, a decision must be judged in terms 
of the time frame allowed for the decision. A quick imperfect decision often will be 
preferable to one which is the "right" decision but is derived too late to be implemented. 
Second, the organization needs to limit personal accountability for the effects of decisions. 
Except in cases of direct culpability because of individual carelessness or lack of effort, the 
decisions made should be seen as a product of the decision system and therefore a 
responsibility of the organization and not just of the individual. 

Given the current nature of decision practices in most countries, the latter requirement 
will not be fully realized. The use of individual scapegoats to deflect criticism from the 
organizational unit (or from the government) remains a bureaucratically and politically 
popular technique in both developing and developed nations. 

However, to the extent that data and logic allow past decisions to be justified and the 
decision pro'cess to be democratized, it will be more difficult to assign fault for bad results 
to a single individual or unit. This process of facilitating decisionmaking is itself facihtated 
if senior administrators and, in the case of government, politicians also have been exposed 
to the benefits of using data and logic in the ways proposed here. 

Finally, as with all education, training of managers is not a finite but a recurrent (if not 
constant) activity. The information system must be designed so that increased training 
allows the managers to alter their information demands and so that changes in data 
availability or information technology can encourage new forms of training. Information 
quality and decision-making quality should be allowed to improve concomitanUy; an 
imbalance between the two will result in a negation of the quality of either kind. 



III. CONSTRAINTS ON AND FACILITATORS OFEFFICIENCY ANALYSIS USE 
IN EMIS DEVELOPMENT 

To understand the policy relevance of efficiency analysis it is necessary first to 
understand the role that efficiency analysis can play in the EMIS operations of an 
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educational institution or system. Fouralternative situations may occur from the collection 
and assimilation of educational data within an efficiency framework. First, the analysis 
may be used to evaluate existing policies and practices and to develop new ones. Second, 
it may be used to support policies and practices tiiat ahready have been determined bureau- 
cratically or politically. In this second instance, efficiency analysis use would not affect 
educational activities immediately or directly. When results reinforced what the senior 
decisionmakers wished to do anyway, Uie results would be used. In such cases, all data and 
analyses are valued not in terms of their ability to inform new decisions but in terms of their 
ability to justify existing ones. 

Third, efficiency analysis and data may have no effect at the level of policy or practice 
oUier than to be added to the educational data base. In this situation, senior decisionmakers 
are unconcerned with the data results, whether the results are favorable or not However, 
the data and analyses still have the potential to affect individuals' perceptions at the 
technical and lower adm inistrative levels of the institution or system. Fourth, the data and 
analyses may be ignored at both the decisionmaker : nd technician levels. This situation 
often will lead to the discontinuance of efficiency analysis and of the supportive data 
collection and assimilation functions. However, tiie inertiaof some EMIS structures is such 
that it is not impossible tiiat efficiency data, like much current education data, will continue 
to be collected (and even reported) witiiout any evidence of its being ^plied to any 
purposeful outcome. 

The question of which of tiiese four situations will occur in a given country or educa- 
tional institution is a function of the relative strengtii of the constraints on, versus the 
facilitators of, efficiency analysis witiiin tiie EMIS. Ultimately, all data and information 
use will be determined by the characteristics of suitability (relevance to perceived issues), 
understandability (the capacity of decisionmakers to comprehend the data and informa- 
tion) , accuracy (Uie degree to which the data and information correspond to otiier indicatoi^ 
of reality, internal consistency, and past predictive value), and timeliness (tiie temporal 
correspondence of availability witii need). For efficiency analysis results, four main 
constraints and four main facilitators have been identified that will affect the perceptions 
of these characteristics by decisionmakers. 

A. Constraints 

The first and most serious constraint on the use of efficiency analysis witiiin an EMIS 
is tiie lack of understanding by ^isionmakers of the terms, concepts, and decision 
criteria used in such analysis. Altiiough based on logical decisionmaking models, 
efficiency analysis appears intimidating to Uiose unfamiliar witii its specialized terminol- 
ogy. Only through decisionmaker training, of tiie types described above, can tiiis constraint 
be overcome. Obviously, to achieve tiie desired participation in training one will have to 
overcome reluctance on tiie part of decisionmakers to engage in such training. The high 
opportunity costs of tiicir time and tiicir own initial inability to value tiic possible benefits 
will discourage the willingness of some individuals to participate in such training. 
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To overcome this second-order constraint will involve a marketing effort on the part of 
the agencies or organizations that desire such training. This marketing effort will be 
supported bv some of the facilitators to be discussed later. However, in developing nations, 
national planning units and donor agencies can combine efforts to encourage greater 
recepuveness to the training opportunity; the first to produce the demand on EMS 
operauons to use efficiency analysis and the second to provide scarce resources and trammg 
opportuniUes. Increasing the abUity of decisionmakers to understand efficiency analysis 
will enhance greaUy the probabUity of its incorporation within the EMIS and its use in 
determining future policies and practices. 

The second major constraint on efficiency analysis within an EMIS r. the cost of data 
collection and assimilation. Those efficiency measures that depend upon qualitative or 
observational techniques will be especially hard to jusufy for systems with a shortage of 
data system resources. The soluUon is that each EMIS must begin with a core set of 
measures emphasizing those cost and effecuveness indicators that are affordable withm its 
budget The iniUal emphasis should be on the easily quantifiable and immediate versus the 
qualitative and distant. But it should be recognized that this system is a foundation for the 
EMIS not the capstone. As soon as possible, a set of recurrent, observational studies of 
specific problem areas should be initiated as a parallel activity to the basic educational 
census. 

The core EMIS information can be supplemented further by special studies of cost and 
effects of programs that require immediate attention but do not require or justify recurrent 
study. Project analyses would be an exemplary case of such studies. 

In every case, a cost-utility analysis must underlie each decision to add. maintain, or 
delete a form of data or analysis within the EMIS. The administrative head of the EMIS. 
supported by an advisory committee consisting of information technicians and educational 
decisionmakers, ultimately must be the locus of responsibility for this cost-utility analysis. 
Such analyses also can be the basis for requests for additional funding of the EMIS. 

The tiiird constraint on the use of efficiency analysis within an EMIS is the concern over 
suitability, accuracy, and timeliness of efficiency information . As the discussion on cost 
and effectiveness measures illustrated in detail, the more suitable the efficiency measure, 
the more problems it may pose in terms of accuracy or timeliness. The closer a cost or 
effectiveness measure approaches a conceptual ideal the more difficult it may be to 
operationalize and to measure accurately and the more time its collection and assimilation 
are likely to require. The result of this condition is to reduce the ability of efficiency 
analysts to justify their results to other educationalists. 

Once again there is no facile solution. The analyst must balance the utility of a more 
sophisticated and precise measure of cost or effectiveness against the disadvantages in 
termsof (1) financial expenditures on collection, validation, processing, and interpretation 
and (2) time delay from the request for information until it is available. Efficiency analysis 
is unique in this regard. It not only provides a basis for organizing an EMIS by specifying 
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types of data thai ideally should be selected, it also provides criteria for devolving from the 
ideal to what is practical given an EMIS organization's human, physical (equipment and 
facilities), and financial resources. 

The final major constraint on efficiency analysis is the concern over redistribution of 
organizational power. This is a special case of the general data phenomenon that as data 
increase, those who control and/or understand the data gain influence. This can be 
manifested in terms of both a horizontal and vertical restructuring of po>v'er. Horizontally, 
a director of educational statistics may increase his or her influence at the expense of 
directors of other "line" divisions within an organization (the latter would be the heads of 
such units as primary education, teacher training, and vocational/technical programs). If 
the other directors do not have the skill to assess data and to summarize and interpret data 
reports, these officials will have a less effective impact on the decision process within their 
organization. 

Th/ './suit may be that the interest of the units headed by these directors will be less well 
represented. In the short run this could lead to greater dependence on those quantitative 
measures that ire the common products of statistical units; in the long run the effects will 
be to undervalue all experiential and qualitative insight and to elevate the head of the data 
unit to a position of "f among equals" if not toade facto superiority over the other 
directors. 

Vertical r iignments of power can be caused to the extent that data— and especially 
data genera, ^y efficiency analysis— is understood by junior administrators but less well 
understood by their superiors. The senior officials, if they are notable to ignore such data, 
may bee increasingly deferential if not overtly dependent on their subordinates to 
explain the data and analyses and for guidance in extracting recommendations. This 
process of dependence may be gradual but will culminate in the creation of a technocratic 
level within the organization that has an influence on the final decisions that far exceeds 
that indicated by the placement of the technocrats within the organizational chart 

R. Facilitators 

To offset the influences of these constraints, four specific facilitators of increased use of 
efficiency analysis have been identified. The first, and least subtle, is the self-interest of 
the units who collect and assimilate data. The vested interest of such units is to increase 
the demand for and use of their production. These units, and their personnel, normally will 
be a constant source of lobbying efforts to promote a greater role for all data in the 
educational unit's or system 's decisionmaking processes. There is no stronger indicator of 
bad management than a statistics unit that passively awaits requests for data or suggestions 
of new types of data that may be generated. While one appropriately may be suspicious of 
excessive self-promotional zeal on the part of data units, excessive passivity is an issue of 
even greater concern. The ideal situation is a data unit that hopes to expand its influence 
by improving the characteristics (applicability, understandability. accuracy, and timeli- 
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ncss) of the educational information it produces while showing appropriate responsibility 
in terms of data costs. 

A more generic facilitator within educational organizaUons of the use of efficiency 
analysis is the desire for managers to have a means to depersonalize the decisionmak- 
ing process. As was explained earlier, bureaucratic systems have evolved patterns of 
individual responsibility for bad decisions as a means of protecting the credibility of the 
overall bureaucracy. However, the individual decisionmakers can attempt to protect 
themselves only by presenting evidence that they based their decisions on accepted data 
and decision criteria. Thus, the increased availability of data facilitates this depersonaliza- 
tion of culpability. 

The third facilitator is related to the above in that it is characterized by the tendency of 
decisionmakers to promote creation of a common data base for decisions. The advan- 
tage for such a common data base is that it facilitates more general participation in decisions 
while focusing the discussion on data interpretation. Rather than having five differem 
opinions on the probable number of students or of the ratio of boys to girls, the debate can 
concentrate on the meaning for policy and practice of the accepted figures on enrollments 
and gender proportions. 

Finally an importam impetus toward efficiency analysis specifically and better data 
generally Is the need for the EMIS to attain or maintain parity with other information 
systems This need can be formalized by government as in the case where the agency 
responsible for national planning sets data requirements for all administraUve^un.ts in 
government. Alternatively, the pressure may be less formal but equally powerful if the 
education unit or ministry finds itself at a disadvantage in policy or finance debates because 

of the lack of persuasive efficiency data comparable to that possessed by compcung units 
or ministries. The international agencies, especially UNESCO, have had a role m the past 
in promoting standardized data collection. If such agencies increased the relevance of these 
standard systems by reorganizing them around the efficiency principles, the systems could 
be disseminated widely with a significant positive effect on individual nauonal data 
operations.* 

The net effect of the aforememioncd constraints and facilitators of efficiency analysis 
within EMIS structures will vary from nation to naUon and even among educational units 
within a single nation. However, the overall trend is clearly discernible: the educational 
data base is increasing in quantity and quality and so are the informauon processing 
systems The ultimate constraim and facilitator is the nature of human capaciues: the 



The potential for such reorganization was refiectcd by the degree of interest in the recent 
OECD conference on educational indicators (Washington, D.C., November 3-6, 1987). 
The range of views presented are suggested by the papers presented by the U.S. 
Department of Education. C.E. Finn. Jr.. T.N. Postlethwaite. A. Purvis, and K. Eidc. 
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capacity of the information technicians to improve the four characteristics of their data and 
its affordability and the capacity of decisionmakers to use the data effectively. Ultimately, 
these capacities will determine the structure of the overall EMIS and the role of efficiency 
measures and indicators within it 

In the next section a brief summary of the earlier discussions will be presented. This will 
be followed by a list of general recommendations that deal with how national governments 
and donor agencies can increase the role of efficiency analysis in the review and 
formulation of educational practices and policies so as to promote greater individual 
benefits and enhanced systemic efficiency. 
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CHAPTER SEVEN 
SUMMARY AND PROPOSALS 



I. SUMMARY 

For a variety of cultural and political reasons most education and training programs have 
been organized as public or private non-profit activities. The goals of these activities are 
rarely clearly specified, if even defined, and, in any case, may be expected to vary 
depending upon the interests of the multiple stakeholders in the human resource develop- 
ment enterprise. Moreover, there remains a limited understanding of the objective 
functional relationships that exist within and among the four stages of educational 
production— inputs, process, outputs, and outcomes. 

The result of the imprecision in the knowledge of goals and the inadequacy of the 
understanding of the individual educational variables and their compound relationships is 
to make management of education an exceedingly difficult task for the student and parent, 
for the institutional administrator, and for the public planner or policymaker. Because most 
educational decisionmaking is conducted in a context of diffuse and uncertain incentives, 
educational management has been characterized by a lack of consensus as to goals and 
standards. All educational managers operate in an environment that subjects them to short- 
term political and social pressures that may compromise their attempts to achieve long- 
term resource utilization, socialization, and human capi J development goals. The current 
problem in education, in both developed and developing nations, is not just the present state 
of systemic ineffectiveness in the accomplishment of goals and the common inefficiency 
in the use of public and private resources. The greatest source of concern should be that 
there are so few current incentives that will encourage managers and users of education to 
improv<^ the system and its individual institutions. 

The concern over this issue is great for two reasons. First, education and human resource 
activities are. next to police and defense operations, the single largest category of public 
expenditure in most countries and an increasingly important part of private expenditure m 
many countries. The current size of the expenditure on education will be under great 
pressure in the remainder of this century both from population increases and demands for 
more and better trained workers. The extent of the social demand effect is mdicated by 
statisucs such as those from Africa that project that, in the next two decades. 1 10 million 
new students will have to be absorbed by educaUonal systems that already have been 
overextended by the 50 million new students of the last two decades.* 



* Durujki (1978) iiotes that: "To force the pace of educational development leads to one 
absolute certainty. Standards of scholasuc attainment begin to fall and continue in a 
downward trend until, paradoxically, education for all becomes education for none." 
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The second source of concern relates to the potential effect of continued educational 
inefficiency on the operation of the economies and societies in the developing world. Much 
of the economic, social, and political progress of the last quarter-century exists on an 
extremely fragile base. More than any other social institution, education will determine 
whether that base is reinforced or eroded. Improvements in the quality and the equality of 
educational oppx)rtunities in the developing world can be assured only by efficiency 
enhancement activities. 

Every human resource system has three financing alternatives when faced with increas- 
ing social and economic demand (Windham, 1986). These are: (1) to obtain new levels and 
sources of funds; (2) to accept poorer quality and/or reduced access; and (3) to increase the 
efficiency with which existing and future resources are used. The first alternative will not 
be available in many countries and almost all developing nations will find the increase in 
resources over the next quarter-century to be less than commensurate with the demands 
placed on the educational system. The second alternative is explicitly unacceptable but 
implicitly utilizedby an increasing number of nations who are politically pressured to allow 
social demand for education to expand beyond the level where quality of instruction can be 
maintained or equality in access promoted further. 

If the first alternative is unavailable and the second should be unacceptable, then 
efficiency enhancement activities cease to be simply a means of controlling costs and 
become instead the central organizing operations for the planning, delivery, and evaluation 
of education and training programs. Only by emphasizing more efficient use of present and 
future resources (financial and human) can educational systems provide more and/or better 
opportunities for personal and social improvement. A delay in implementing efficiency 
reforms will not simply increase the problem, it will reduce significantly the probability 
that the problem can be solved. The risk is not just that funds will be wasted or that 
government budgets will be strained; a failure of the education system that is concomitant 
with the current high level of social and economic aspirations of parents and children can 
lead only to economic disfunction and social distress. These dire warnings are not the 
products of the generic pessimism of the economist's "dismal science;" rather, they are a 
simple extension of phenomena that already may be perceived in the large majority of 
developing nations in Asia, Africa, and Latin America. 

Also, it should be stressed that it will not be appropriate for the developing nations to 
await models and examples from the developed nations before beginning efficiency 
reforms. The wealthier a nation, the more foolish and wasteful it can afford to be. The 
inefficiencies in education in developed nations are more politically tolerable because 
these nations both have more financial resources (current and projected) with which to 
disguise their inefficiency and lower social demand pressures (bccaus.'^ of slower popula- 
tion growth) that would expose the inefficiencies. 

Developing nations will have to establish the examples of efficiency enhancement that 
eventually will be copied by the developed nations. This makes the marketing of efficiency 
proposals more difficult since developing nations are more accustomed to importing rather 
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than exporting social cxpcrimcnis. Also, the dcvclopbg nations have a legitimate basis in 
their historical experiences for distrusting attempts by developed nations to test reforms in 
the developing world that they are not willing to test in their own societies. These barriers 
to efficiency enhancement are real but must be overcome. Most difficult, these barriers 
must be overcome before the educational situation deteriorates beyond what even effi- 
ciency reform can do to salvage it 

The major purpose of this monograph has been to provide a con text within which debate, 
planning, and monitoring of efficiency reforms can lake place. In addition to inu-oducing 
the economic terms and concepts related to educational production and efficiency, an 
attempt has been made to discuss the state of policy analysis concerning many of the 
variables, measures, and standards presented. As noted in the original introduction, this 
presentation has attempted to balance the apparent precision of economic theory with the 
complexity and uncertainty of administrative practice. While the result may involve a 
sacrifice of both some of the more refined aspects of economic theory and the details of 
daily educational administration, some individuals still may question the need for the 
degree of both abstractncss and complexity that remain. The simple fact is that the major 
barrier to efficiency analysis does not lie in mastering the supportive economic concepts. 
These concepts— and the derived terms and models— are generally logical and easy to 
master; anyone not willing to make the effort to master them deserves to be disqualified 
from a major decisionmaking position in education. 

However, the true complexity of efficiency analysis originates in the nature of education 
itself; specifically, the variety of types and levels, the extraordinary variability among 
determinants and effects, and the requirements for subjective judgment conducted within 
acontextofmultiplestakeholderswithdifferingand,attimes,mutuallyexclusivegoalsand 
values. The presentation here has been designed to clarify the use of the economists' 
concepts, terms, and models and to explain the inherent complexity of educational 
decisionmaking and the appropriateness of subjective judgments. Educauonal decision- 
makers cannot avoid responsibility for the judgments they make concerning educational 
costs and effects; however, by using the efficiency tools presented here they can minimize 
the arbitrariness of their decisions and assure themselves of being able to provide a clear, 
if ultimately subjective, rationale for the decisions that have been made. 

The discussion of the research literature has been designed to characterize what 
educational researchers believe they know about educational production and efficiency. 
The criticisms presented of this research have had the purpose of emphasizing the more 
**efficient" alternatives for research designed to promote efficiency enhancement in 
education. Efficiency research must be planned and judged by the same standards of cost 
containment or effect maximization that the researchers apply to the educational system 
itself. 

In summary, the efficicricy concept has been asserted to be a useful metaphor for 
educational analysis even though educational activities have few characteristics that are 
analogous to the technical production systems that originally gave rise to efficiency 
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concepts. Applied to education, the efficiency metaphor cannot be used to create a self- 
regulating, self-sustaining set of controls similar to those of classical competitive economic 
markets. In education, in both the public and private sector, efficiency analysis must be 
incorporated as a device for bureaucratic or individual decisionmaking. In both the 
bureaucratic and individual case, there are three requirements for effective decisionmak- 
ing: (1) training in decisionmaking logic generally and in efficiency analysis specifically; 
(2) improved information on educational costs and effects; and (3) the promotion of sets of 
incentives that encourage the use of both better decisionmaking skills and the improved 
information. 

For individual decisionmaking the incentives already exist in terms of the individual's 
self-interest. These incentives will be increased as more countries choose or are forced to 
implement more user-finjtncing of education. Improved bureaucratic incentives are more 
difficult to generate or promote. However, as the financial and human resource problems 
of nations increase with lime and senior policymakers themselves become more sensitive 
to the issues of efficiency, the bureaucratic incentives for the use of belter decisionmaking 
skills and improved educational information should be realized. 

As this discussion has stressed, the current interest in efficiency issues will not prove to 
be a transient phenomenon in educational planning and management. Although the 
efficiency concepts have sometimes been misapplied and resistance to them — for the 
wrong reasons — continues, the efficiency approach to education offers the most inclusive 
and articulate means of designing and evaluating plans, operations, and proposed reforms 
of education at both the system and the institutional level. Economists and financial 
analysts will have no license to impose their opinions on curriculum specialists, trainers, 
teachers, or administrators but all of these individuals should be under increased responsi- 
bility to present justifications for their activities within the framework of probable costs and 
effects. Such analysis must shift from an ad hoc condition to a prerequisite for considera- 
tion or continuation of an educational activity. A cooperative and supportive relationship 
should develop between the efficiency specialist and the educational professionals with 
residual differences more a matter of variant conclusions than of disparate assumptions. 

It has been asserted here that the efficiency principles are a singularly appropriate means 
for organizing the training of decisionmakers and the design and operation of educational 
management information systems. This was done in full recognition of the limits on the 
proper definition and measurement of many of the efficiency concepts in terms of 
education variables. Also, it has been stated here that the more refined and conceptually 
appropriate an educational efficiency variable becomes, the more difficult the variable will 
be to operaiionalize and to interpret. The conclusion reached is that efficiency analysis is 
a cumulative process that can provide some immediate answers but has its greatest value 
in providing better long-term answers as efficiency information evolves. Supporting this 
point is the equally important assertion that educational efficiency analysis must be based 
on multiple indicators. Multiple indicators of efficiency allow for an internal check of 
quality and intcrpretability and at the same time are responsive to the varying perspectives 
of multiple stakeholders in the educational process. 
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The efficiency approach will require a change in how educational decisionmakers are 
trained and how educational data are collected and used. While tradition and ease of 
collection will always be legitimate considerations, educational management information 
systems must be reoriented more away from tiie interest of data collectors and to tiie needs 
of data users. The principles of efficiency analysis, as discussed here, can help assure that 
this happens. 

The preceding is presented as background for diree major proposals for the restructuring 
of educational management and decisionmaking. The proposals tiiat follow are not revo- 
lutionary but tiiey are designed to accelerate the evolutionary process engendered by tiie 
current fiscal and human resource problems faced by so many countries. 



n. PROPOSALS 

The research, training, and operational alternatives for advancing die role of efficiency 
analysis in educational management have been summarized as Oiree major proposals: 

1 . Training and upgrading of beginning middle-level and senior educational deci- 
sionmakers in decision-making principles and in die necessity for and die means 
of operationalization of efficiency analysis in educational management; 

2. Establishment and monitoring of effectiveness and efficiency benchmarks witiiin 
educational institutions and systems; and 



3. 



Development and maintenance of an educational management information sys- 
tem based on die principals of efficiency analysis. 

While detail in these proposals will depend upon die needs and resources of a specific 
national context, die discussion here wUl stress die generic framework for each of die diree 
proposals. Also, die relationship among die diree proposals will be emphasized. The 
implementation of a single proposal or any pair of proposals will not have a proportional 
impact on improving educational management. The proposals are designed to be consid- 
ered and initiated together. 

The issue of die training and upgrading of educational decisionmakers was raised 
earlier widiin die context of die administrative capacity discussion. While individual de- 
cisionmaking practices of students, parents, and community members in education are of 
equal importance, die greater feasibility for reform lies in dealing widi administrators of die 
educational bureaucracy. In most countries diese individuals control financing and 
personnel decisions and have a significant constraining or facilitating effect on cumculum 
and classroom practice. 

As mentioned earlier, Uiis training should incorporate bodi die general principles of 
improved decisionmaking and die specific concepts of efficiency analysis apphed to 
education. The training program should be organized around four major sets of activities. 
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First, pre-service training for all educational personnel should include an introduction 
to basic efficiency concepts and a justification for their central role in educational decision- 
making. This pre-service training would create acommon basis for discussion of efficiency 
issues among teachers and between teachers and administrators. Second, all school and 
institution or program administrators should receive more advanced training in efficiency 
analysis as a prerequisite or a concomitant requirement to their assuming new administra- 
tive responsibilities. 

Third, regional and central middle-level administrators should receive special detailed 
training in efficiency concepts, decisionmaking, and the use of educational data. Normally, 
this training should take place at the work site and result in the minimal disruption of 
ongoing work responsibilities. When time and other resources permit, training of middle- 
level administrators can involve more extensive three- to six-month courses of full-time 
training. 

Finally, the fourth form of training will involve the most senior decisionmakers in the 
education and related human resource, planning, and finance ministries or agencies. Here, 
the focus is less ou the details of efficiency analysis and more on the justification for 
increasing the focus on efficiency principles in their evaluation of reports and proposals 
prepared by the middle-level administrators subordinate to them. 

Various donor agencies have conducted many administrator training programs and 
some, especially USAID and the World Bank, have promoted efficiency approaches in 
their administrator training programs. The immediate need is for the collection and 
integration of these training experiences as a basis for designing a standard set of training 
curricula for each of the four levels of training. 

A joint effort by donors to assist in the design and implementation of such training can 
be tailored to individual country needs through the participation of host country personnel. 
The training programs should be closely coordinated with those agencies responsible for 
educational data collection and policy formulation. 

The establishment and monitoriRg of efficiency and effectiveness benchmarks 
would appear to be a simple and obvious proposal. However, educational programs and 
projects often proceed without operational criteria by which the program or project can be 
judgged to have succeeded or failed. Understandably, there is a normal bureaucratic 
reluctance to establish performance standards and a preference to state long-term goals and 
even short-term objectives in the generalized language common to national planning 
documents. 

The danger of operating any system without established standards and benchmarks is 
quite serious. Problems ore not normally detected until an ad hoc review or examination 
of the system is conducted. Thus, errors or inefficiencies can continue for a substantial time 
and, most serious of all, become part of the accepted administrative practice of the 
enterprise. Also, ad hoc assessments or evaluations will not lead to the needed reforms 
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unless pan of the reform is the establishment of benchmarks to allow evaluation of the 
reform effort itself. 

Because of the inevitable bureaucratic reluctance to expose programs to review and 
evaluation, reform can only come through a commitment of the most senior officials of a 
system or organization to this new means of operation. Evaluation must not be viewed as 
a forensic exercise designed to identify culpability and assign blame; rather, it should 
become a standard procedure for identifying a means for improving the operation of the 
enterprise and for assigning responsibility for the reform. One of the most difficult tasks 
that will be faced by those advocating efficiency reforms for developing nations will be the 
need to reorient evaluative activities from backward-looking investigations of what went 
wrong to forward-looking examinations of alternative opportunities for improvement. To 
achieve this will require that the reformers overcome aspects of both bureaucratic tradition 
and normal human psychology. 

Any establishment of educational benchmarks must begin with selection of what are 
called "objectively verifiable indicators." These are quantitative measures designed to 
indicate the nature of change, its direction, and extent, A simple example would be female 
enrollment statistics. Over time or from place to place one can compare the change in 
enrollments in terms of both their direction and size, A slighUy more sophisticated measure 
dealing with the same topic would be the percentage of female enrollment. As opposed to 
simple enrollment, the later measure will indicate the change in female enrollments rel ative 
to changes in the enrollments of their male counterparts. 

This example should suggest two further considerations. First, no system of bench- 
marks can be meaningful in evaluation unless the original assessment created baseline 
data to which the later benchmarks may be compared. Even in donor-financed educa- 
tional projects, where great attention normally is given in the planning and decision process 
to establishing verifiable standards, the necessary baseline data collection often does not 
take place during the complex activities of initiating the approved project. Thus, when 
interim evaluation efforts begin, the cvaluators find they have no baseline standard with 
which to compare their benchmarks. A cynical interpretation would be that the purpose of 
discussing objectively verifiable indicators is to obtain project approval not to institute an 
actual monitoring process of project accomplishments. A more generous interpretation 
would be that external authorities, the funding agency, and/or the government unit 
responsible for project implementation must lake greater responsibility for assuring that 
evaluation considerations are part of any project's initial concerns. If this is not done, the 
immediate organizational priorties of project managers will dominate those of their 
administrative superiors who are responsible for the wider concerns of ultimate project 
efficiency. 

The second further consideration is that the comparison of benchmark with baseline 
data is only the beginning, not the end of evaluation. This is Uue in all evaluation but 
is an especially important point in efficiency analysis. Efficiency evaluation will depend 
on muluple indicators, on measures of both costs and effects, and will always require a 
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subjective interpretation of the data before policy conclusions can be reached. It is 
critically impprtant that the efficiency benchmarks not be used to create a mechanistic 
evaluative process wherein an educational institution or regional unit has its success or 
failure measured by a single rate or ratio or even by multiple indicators. The indicators and 
the analysis are not the same phenomenon; the link between the two is the conceptual 
understanding the decisionmakers have of the educational process and the values they 
apply to the data in reaching policy conclusions. 

An inherent part of the policy process should be the periodic review a nd modification 
of benchmark data. As a data system and its users become more mature, more complex 
interaction statistics can be introduced. The actual progress of a system will depend on 
where it begins (in terms of data quality and decisionmaker capacities), the resources made 
available, and the importance assigned by decisionmakers to efficiency considerations. 
While substantial variation will occur from nation to nation. Table Twelve presents three 
possible levels of development the baseline/benchmark system might follow. These stages 
arc not fixed in their detail nor would they necessarily be distinct in their implementation. 
The initial degree of efficiency detail will be a function of the state of development of the 
existing educational data system. 

The benchmark system will be organized around seven sub-categories of data: 

- Student characteristics 

- Teacher and administrator characteristics 

- Curriculum and educational materials 

- Facilities and equipment 

- Attainment and achievement data 

- Education and training outcomes 

- Costs 

In the progression from Level I to Level III the data will increase in coverage, accuracy, 
and intcrprctability (and, if the EMIS is successful, in timeliness). The intcrprctability 
gains will occur in part because of a greater capacity to assimilate data through compari- 
son and contrast of data sets. For example, gender ratios can be combined with teacher 
characteristics by region and across time to provide a basis for discussions of coincidental 
effects and possible causality (e.g., more women teachers may encourage greater atten- 
dance and retention of female students). 

The rate of increase in detail and coverage between levels of development will vary 
among the seven categories of data. In most countries student, teacher, and administrator 
data will be emphasized while in others the focus may be on costs or on facilities and 
equipment utilization. While variation in rates of change in the seven data groups is 
unavoidable it still is a matter of concern. For example, if cost detail exceeds information 
on input quality or on output and outcome effectiveness, this condition can give rise to 
serious efficiency misinterpretations. The goal of the data benchmark system should be to 
emphasize a balanced development across the seven data categories so that companibility 
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TABLE TWELVE 

LEVELS OF DEVELOPMENT FOR SYSTEM OF 
EFHCIENCY-BASED BENCHMARKS 

LEVEL ONE 

L SludeniData 

- Enrollmenl by schcx)l 

- Gender ratios 

- Progression rates (aggregate only) 

2. Teacher Data 

- Distribution by qualifications 

- Distribution by location 

- Student-Teacher ratios 

3. Curriculum/Educational Materials 

- Textbook availability 

- Regional and size-of-place distribution 

4. Facilities/Equipment 

- Number of "complete" schools 

- Students per school 

- Students per class 

5. Attainment/Achievement 

- National examination pass rates 

- Prom*.^tion rates 

6. Outcomes 

- No d^^.ca 

7. Costs 

- Teacher salaries by qualifications 

- Aggregate budget data 

- Cost per student by level of education 

LEVEL TWO (All Level One data plus the following) 

1. Student Data 

- Gender data cross-tabulated with size-of-place 
and region 

- Ethnic distributions 

. Detail by level and type of program 

- Separate repetition and attrition rates 

- Age distributions 

2. Administrator and Teacher Data 

- Qualifications distribution including specializations 

- Age and experience 
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- Distribution by location 

- Students per administrator 

- Turnover rates and incidence 

- Absenteeism 

3. Curriculum/Educational Materials 

- Textbook availability and utilization 

- Availability of support materials 

- Status of curriculum development and dissemination 

4. Facilities/Equipment 

- Facilities utilization by level and type of program 

- Equipment availability 

- Distribution of special use facilities 

5. Attainment/Achievement 

- Examination scores and pass rates cross-tabulated with 
student and school characteristics 

- Attainment distributions by student and school characteristics 

- Promotion rates by student and school characteristics 

6. Outcomes 

- Earnings data from public employment 

- Employment data (aggregate) by level of education 

- Tracer studies of secondary school and higher education graduates 

7. Costs 

- Ingredients-approach cost calculated for each level and 
type of program 

- Unit and cycle costs for all programs 

LEVEL THREE (All Level One and Two dau plus the following) 

1. Student Data 

- Subject or course specializations 

~ Attitudinal and behavioral measures 

- Time utilization 

2. Administrator and Teacher Data 

- Time utilization 

- Training needs 

- Interaction with community 

- Job satisfaction 

3. Curriculum/Educational Materials 

- Knowledge of curriculum by administrators and teachers 

- Users' evaluations of curriculum and materials 

- Evaluation of alternative instructional technologies 

4. Facilities/Equipment 

- Equipment utilization 

- Needs analysis 

- Maintenance and replacement projections 
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5. Attainment/Achievement 

- Determinants of educational outputs 
. Determinants of inequalities 

- Analysis of high- and low-achieving schools 

6. Outcomes 

- Net present value estimates by level and type of education 

- Studies of graduate attitudes and behaviors 

~ Job search rates by level and type of graduate 

7. Costs 

- Detailed cost analyses of major programs and alternative technologies 

- Cost projections by level and type of education 

in detail, coverage, and accuracy promote improved interpretability for policy purposes of 
the total data system. 

Data detail also may be expected to vary by level and type of education or training. 
Because of the increased level of operaUng expenses, one may expect a greater availabil- 
ity of cost detail to emerge in the higher education and vocational training subsectors. 
Because of the political and social importance of concerns with basic educational 
opportunity, measures of gender, ethnic, size of place, and regional equity in access and 
retention may be collected in greater detail at the preprimary and primary educational 
levels. However, even these patterns of data detail by level and type of education will vary 
from country to country. 

The timing of the benchmark system illustrated in Table Twelve also may be expected 
to vary according to resource availability. Benchmark data will depend most heavily on the 
annual educational census. Special data collections will coincide with the mid-term and 
inter-term national planning cycles. In addiuon to these major acUviUes. the benchmark 
system can gain supplementary detail from special studies conducted by government and/ 
or donor personnel as part of project planning and evaluation activities. As the earlier 
discussion of educational process measures illustrated, the introducUon of recunent 
"special studies" on education (involving both observational and longitudinal studies) also 
can be a factor in enriching data detail available for educational decisionmakers. 

The level of development of the benchmark system should determine the nature of the 
aforementioned decisionmaker training programs. However, the relationship between data 
development and training should be such that the training anUcipates the increases in future 
data sophistication. An extra benefit of training that assumes or anticipates improved data 
is that it will give rise to a demand for such data by facilitating its eventual use. 

The creation and development of an efficiency benchmark system have an enormous 
potential for increasing the sophistication and professionalism of educational decisionmak- 
ing processes in developing nations. However, host country personnel are likely to remain 
skeptical about the benefits of efficiency benchmarks un til they see donor projects applying 
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such benchmarks to themselves. It is a matter of special concern when government and 
donor projects that proselytize efficiency enhancement operate without clear standards for 
their own effectiveness or cost and effectiveness benchmarks for their own operations. 

The final proposal derived from this discussion of efficiency analysis is to advocate 
creation of an efTiciency-based educational management information system. The 
major types of data to be included in such an EMIS are indicated in Table Thirteen. The 
data types are organized in terms of the four parts of the educational production process. 
The development of the efficiency-based EMIS will parallel the three levels discussed 
above for the efficiency benchmarics. Once again, the state of data development at any 
point in time . jl a given nation will be a function of the financial and human resources 
devoted to the EMIS. In turn, the amount and quality of these resources will be a function 
of the policy importance assigned to the joint efficiency tasks of cost containment and 
effectiveness maximization by politicians and by senior policymakers in the education and 
human resource sector. 

The efficiency-based EMIS can only be properly understood within the structure of 
decisionmaking that is common to the public and private education sectors. This 
decisionmaking process has five main stages. First is the analysis of the current status of 
existing policies and practices. Second is the specification of current plans. Third is the 
identification of currenUy unmet needs and of emerging problems. Fourth is the drafting 
of proposals for new policies, practices, or plans. Fifth is the derivation of the required 
changes in organizational structures and incentives and in the quantity and quality of 
resources. The last is basically an analysis of the financial consequences of the proposed 
modification in policies or practices or of the implementation of newly planned educational 
initiatives. 

The EMIS, to be efficient itself, must be able to provide decisionmakers with the data, 
information, and even analysis that is required during each of these five stages. To fulfill 
this responsibility, seven steps will need to be followed in evolving from the existing data 
system to a fully operational, efficiency-based EMIS. These steps are the following: 

1. Assess current data collection and assimilation activities in terms of their 
coverage relative to users' expressed needs and EMIS specialists' recommenda- 
tions and the adequacy of current levels of data quality (accuracy, timeliness, and 
interpretability) given present and projected uses of the data; 

2. Identify priority needs for new data by comparing projected requirements for data 
with current status and planned changes; 

3. Conduct a cost analysis of new data ingredients with an emphasis on marginal 
costs of differing data amounts, types, and quality; 

4. After using the foregoing to justify supplemental budget requests, analyzehow the 
budget for the EMIS should be allocated; 
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TABLE THIRTEEN 

SUMMARY OF INDICATORS FOR AN 
EFHCIENCY-BASED EMIS SYSTEM 

L INPUTS 

A. Teacher Characteristics 

- Fonnal educational attainment 

- Teacher training attainment 

- Age/experience 

- Attrition/turnover 

- Subject specialization 

- Ethnicity/nationality 

- Subject mastery 

- Verbal ability 

- Attitudes 

- Availability measures 

B. Facilities 

- School size 

- Classroom size 

- Students per school 

- Students per class 

- Classrooms per school 

- Classes per classroom 

- Availability of special-use facilities 

- Utilization of special-use facilities 

- Condition of facilities 

C. Equipment 

- Availability 

- Utilization 

- Condition 

D. Curriculum/Educational Materials 

- Availability of textbook and support materials 

- Utilization of textbook and support materials 

- Articulation of curriculum 

- Dissemination of curriculum 

E. Administrative Capacity 

- Educational attainment 
~ Administrative training 

- Age/experience 

- Organizational context and incentives 
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n. PROCESS 

A. Administrative Behavior 

- Frequency, extent and purpose of external ' 
administrative visits 

- Frequency, extent, and purpose of internal 
administrative visits 

- Nature, frequency, and result of contact with community 

B. Teacher Time Allocations 

- Administrative tasks 

- Instructional tasks 

1. Preparation 

2. Instruction 

3. Review 

4. Remediation 

- Monitoring and evaluation 

C. Student Time Allocations 

- Time on-task 

L Interaction with teacher 

2. Interaction with peers 

3. Interaction with materials and equipment 

- Time off-task 

in. OUTPUTS 

A. Attainment 

- Progression rates 

- Attrition rates 

- Repetition rates 

B. Achievement 

- Examination results 

1. Absolute levels 

2. Averages 

3. Scores relative to other groups 

4. Mastery levels 

5. Achievement gains 

6. Effect sizes 

- School grades 

- Attitudes and behaviors (to be specified and measured for each form) 

C. Equity Effects 

- Range 

- Quartile deviation 

- Mean deviation 

- Standard deviation 
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- Distribution among criterion levels 

- Lorenz curves 

« Gini coefficients 

- Group differences 

OUTCOMES 

A. Admission to further study 

B. Achievement in furtlier study 

C. Employment 

- Initial occupational choice 

- Lifetime occupational choice 

- Aggregate employment rates 

1. Level 

2. Rate and direction of change 

- Job search periods 
L Extent 

2. Results 

D. Earnings 

- Initial 

- Lifetime probabilities 

- Hedonic and equalizing effects 

E. Attitudes and Behaviors 

- Social responsibility 

- Social views and opinions 
Political orientation 

- ConsuiTier behavior 

F. Externalities 

- Increased social mobility and social inclusion 

- Change in distribution of earnings and income 

- Changes in personal values 

- Improved political participation 

- Reduced unemployment 

- Improved mix of manpower skills 

- Enhanced productivity of physical capital 

- Increased quantity and quality of research 
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5. Do annual follow-up analysis of actual data use and identify parts of the EMIS that 
are underutilized and develop appropriate remedial systems (encouraging more 
use of the data by social marketing, reducing frequency or extensiveness of data 
collection, or considering termination of data collection); 

6. Develop and implement a means for promoting incorporation of special studies, 
recurrent analyses, and products of project or program assessments and evalu- 
ations into the EMIS structure; and 

7. Develop and implement a dissemination plan for the full EMIS that will include 
the aspects of promoting improved training for educational decisionmakers and 
establishing efficiency-based benchmarics for all major educational projects or 
programs. 

Once initiated, this seven step cycle should become a recurrent process with ongoing 
considerations of the cost versus effectiveness of alternative data forms, data quality, report 
formats, analytical approaches, and dissemination strategies. The maintenance and 
improvement of the EMIS requires a proactive stance on the part of the EMIS professional 
staff and administrators. 

The status of EMIS development will be the ultimate determinant of the detailed nature 
of the two other proposals made here for decisionmaker training and establishment of 
efficiency-based benchmarks. Neither of the other proposals will be any more successful 
than the EMIS system permits and encourages it to be. It was asserted earlier that for full 
effectiveness these three proposals must be considered as aspects of a single strategy. At 
the heart of this strategy, however, is the assumed availability of efficiency data. Thus the 
EMIS proposal can be fully justified only if both the training and benchmark proposals are 
implemented concomitantly; however, if a choice must be made or a priority aligned, the 
emphasis must be on design, implementation, and proper management of an efficiency- 
based EMIS. 

For donors, these three proposals are congruent with three major strategies currently 
pursued in the education and human resource sector. These are the support for cost 
containment, the facilitixtion of widened financial responsibility and greater individual 
decisionmaking, and the promotion of bureaucratic decentraluation of responsibility and 
authority. The efficiency enhancement proposals presented here are convergent or directly 
complementary to each of these and the increased efficiency of the education and training 
activities of the nation will be systematically supportive of the goals of these donor 
strategies. More important, the efficiency enhancement approach will allow individual 
nations wider alternatives for what can be done for and by their citizens both within and 
outside of the education sector. 

In the poorest countries the efficiency approach will help stave off the most dire 
consequencesof fiscal constraints and accelerating social demand. In the more advantaged 
nations the efficiency approach can mean tlie difference between a degeneration to 
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educational inadequacy and a progression to educational significance in affecung social 
and individual development The ultimate product of all efficiency reforms will be judged 
finally by what happens in the classroom and in the individual student's success or 
frustration in learning. The purpose of the macro-oriented proposals here for training, 
benchmarks, and an EMIS based on efficiency analysis is to promote long-term, micro- 
educational improvements at the level of the school and classroom. The ulumate goal is 
to assure efficient classrooms— classrooms efficient in providing cogniUve and noncogni- 
tive learning opportunities in an equitable manner. 

By promoting the better use of existing resources and improved plans for procuring and 
uulizing future resources, education can be transformed from a teacher employment and 
student containment system to the human resource development system that both the 
producers and users of the system want it to be. Without efficiency standards educaUonal 
programs have no clear incentives to promote their success. The proper use of efficiency 
standards in educational management will promote improved accountability of administra- 
tors and more effecuve utilization of all resources. By avoiding mechanistic and pseudo- 
obiecuve approaches and by accepting the proper role for subjecuve judgment in educa- 
tional decisionmaking and debate, efficiency enhancement ultimately will lead to the 
enhancement of life chances for individual students, of greater professional saUsfacUon 
among teachers and administrators, and of expanded social and economic development 
opportunities for the nation. 

The need now is to proceed with the debate, in a country-specific or even region-specific 
context as to what measures, indicators, and standards of efficiency are suitable for each 
level and type of education and training. Recurrent assessment of these efficiency issues 
will assure that the debate over educational efficiency encompasses concerns with the 
widest possible range of goals and means of the educational process. As the debate 
continues, one can not just hope, but expect, that the better quesuons asked of education 
now will yield better answers in the future. 
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STATISTICAL MEASUREMENT OF EQUALITY 



In this appendix a brief discussion will be presented of some of the alternative statistical 
measures that can be used to quantify the distribution of educational outputs or outcomes. 
As noted in the text, however, these statistical measures of equality can be interpreted in 
terms of equity ("fairness") only by the application of value judgements to the quantitative 
indicators. 

The first of the statistical measures of equality is the range of the distribution. Simply 
defined, the range is the difference between the largest and smallest values in a distribution. 
While useful in comparing variability between or among groups, the fact that range deals 
only with the extreme values of a distribution makes it an unreliable indicator for 
distributions that involve a small number of observations. However, even in sets that 
involve a large number of observations, a single extreme value (called an "outlyer") can 
cause the range measure to misrepresent the extent of the actual variation. For example, 
one could have two distributions of achievement scores with the identical range values of 
20 to 100. However, in one distribution, achievement scores could be spread equally across 
the distribution while in the second, one person could have scored 20, another person could 
have scored 100, and all of the other persons could have scored between 65 and 70. The 
range measure is useful in identifying extreme values but, by itself, does not serve as an 
adequate indicator of the underiying distribution between the extreme values. 

The quartile deviation measure of variabiUty attempts to correct for some of this 
weakness in the range measure; the quartile deviation is equal to one- half the distance 
between the 25 th and 75th percentiles in a frequency distribution. The 25 th percentile (first 
quartile) of the distribution is that value below which 25 percent of all values lie. Similarly, 
the 75th percentile (third quartile) is that value below which 75 percent of all values lie (and 
above which 25 percent of all values lie). The quartile deviation measure emphasizes the 
50 percent of scores that surround the median (the second quartile). Since it measures the 
average distance of the quartile points from the median, it is a better measure of score 
density than is the range. Also, when a distribution is asymmetrical ("skewed'*) the 
comparison of the quartile deviation measure with the median can indicate the direction 
and amount of skewness. 

For example, assume a distribution in which the first and third quartile values are 30 and 
60 respectively and the median (second quartile) value is 50. The quartile deviation 
measure is 45 which indicates a probable negative skewness (the mean value for the middle 
50 percent of the distribution normally will be to the left of the median value since the range 
of sub-median values— 30 to 50— is greater than the rangeof super-median values— 50 
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APPENDIX FIGURE ONE 

SKEWNESS AND NORMALITY IN DISTRIBUTIONS OF CHARACTERISTICS 

FREQUENCY Negative Skewaess 




0 mean median CHARACTERISTIC 

FREQUENCY Positive Skcwaess 




0 median mean CHARACTERISTIC 
FREQUENCY Normal Distributloa 




0 mean=mcdian CHARACTERISTIC 
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to 60). If the median value were below 45, positive skewness for the middle part of the 
distribution probably would exist* 

Appendix Figure One indicates the shape of three distributions: one with negative 
skewness, one with positive skewness, and one with a normal distribution (no skewness). 
Note that in the case of the normal distribution the quanile deviation measure, the mean, 
and the median are all identical. 

Both the range and quartile deviation measures are unsatisfactory for many uses because 
they ignore a substantial number of the values in the distribution (the range ignores all but 
two values and the quanile deviation measure ignores fifty percent of the values). To 
improve on these estimates of variability, two further statistical measures have been 
developed: the mean deviation and the standard deviation. The mean deviation is the 
mean of all of the deviations of individual values in a distribution from a measure of central 
tendency (usually the arithmetic mean although it could also be the median or mode). For 
example, if you have a set of five scores 15,12,10,8, and 5, the mean is obviously 10: 
[15+12+10+8+51 . 



The deviation of the individual values from the mean are +5 ,+2,0,-2, and -5. Summing 
the deviations without regard to positive or negative sign gives one 14; dividing the sum 
of the deviation by 5 (calculating the mean of the deviations from the mean of the 
distribution) gives 2.8 as the mean deviation. In a normal distribution the mean deviation 
will demark the middle 57.5 percent of all values. A large mean deviation value implies 
greater variability around the measure of central tendency and a smaller value indicates less 
variability. It is important to note that the mean deviation is based on all values in a 
distribution and that all values are weighted equally. 

The standard deviation is the most common measure of variability used in research. It is 
like the mean deviation in that it incorporates all values of the distribution but, unlike the 
mean deviation, it weights extreme values more heavily than others. This difference in 
weighing results from tlie fact that calculation of the standard deviation involves taking the 
square root of the mean of the square of all deviations. In our earlier example, the deviation 



* Because the interquartile generation of the quartile deviation measure involves only the 
middle fifty percent of values, it theoretically is possible that the probable skewness 
discussed here may not be reflected in overall measures of the distribution. For example, 
the quartile deviation measure could equal 45 and the median equal 50 (as in the first case 
given here) but the mean of the full distribution could be greater rather than less than the 
median because of a higher incidence of extreme values in the top quartile as opposed 
to the bottom quartile. 
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values of 5;2,0,-2, and -5 would become 25 .4.0.4. and 25; the mean of these squares would 
be 1 1 .6: (5S) and the standard deviation would be 3.43 (the square root of 1 1 .6). The 



standard deviation is the measure of variability with greatest reliability (it is less affected 
by sampling errors) and offers a balance between the range's emphasis or extremes and the 
quartile deviation and mean deviation emphasis on central values. In a normal distribution 
a single standard deviation above and below the mean will delimit the middle 68.26 percent 
of the distribution. Since the standard deviation also is a required component of correlation 
analysis, it has achieved the aforementioned position of predominance in research. How- 
ever, all the measures of inequality mentioned here have legitimate uses and the selection 
of a single statistical measure must be determined by the nature of the underlying 
distribution and the policy issues one wishes to address. 

A more mundane but. at times, appropriate measure of inequality is simply lo study a 
distribution in terms of the incidence (in numbers or percentages) of values relative to 
certain criteria levels. In education, for example, it is common for teachers lo assign letter 
grades for certain test score levels. One example would be the following: 



Obviously, the linkage of letter grade to score is arbitrary (although some teachers may 
determine the numerical values through an assumption that the realized test scores will 
approximate values from a normal distribution) as are the parenthetical value statements 
next to the letter grades. Sometimes, test results or other scores are divisible into only two 
classes — pass or fail — with mastery scoring being a special case of such a binary scale. 

The point is that once a criteria is created, the distribution of values within the criteria 
categories becomes a legitimate measure of variation. For example, if a school has a 70 
percent pass rate this value can be contrasted with a school goal, past school pcrfomance, 
or the performance of other schools. It is important to recognize that measures of variabi 1 ily 
in such standards of achievement involve an explicit use of subjective judgement. While 
all of the equality measures discussed here will require subjective interpretation when 
applied to policy determination or evaluation, the interpretation of assigned letter grades 
and similar standards will require subjective interpretation of a measure that is itself 
subjectively determined. This docs not disqualify the use of these standards as indicators 
of educational effectiveness but it docs suggest that special caution be applied when using 
them. 



A 
B 
C 
D 
F 



(Excellent) 
(Above Average) 
(Average) 
(Below Average) 
(Failure) 



94 
86 
66 
55 
0 



100 
93 
85 
65 
54 
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The next measure of inequality is a diagrammatic rather than statistical one: the Lorenz 
curve. Lorenz curves indicate the cumulative incidence of some characteristic relative to 
the cumulative incidence of the units of observation. Originally derived to study income 
or earnings inequality, the Lorenz curve compared the cumulative incidence of income or 
earnings with the cumulative incidence of population. As indicated in Appendix Figure 
Two, the cumulative percent of income is measured on the vertical axis and the cumulative 
percent of population is measured on the horizontal. It is common to measure the 
population incidence from poorest to wealthiest as one moves from left to right on the 
horizontal axis. Thus, the Lorenz curve of a distribution must always fall on or below the 
diagonal (the curve would be on the diagonal only in cases of absolute income equality) and 
be concave to the diagonal. 

In brief, the Lorenz curve indicates the percent of total income held by various units of 
the population. For example, except in cases of absolute equality, the poorest 10 percent 
of the population (on the extreme left of tlie horizontal axis) must have less than 10 percent 
of all income and the wealthiest 10 percent (on the extreme right) must have more than 10 
percent. The extremes of the curve must touch the opposite diagonal comers of the 
rectangle since zero percent of the population will have zero percent of the income and 1 00 
percent of the population must have 100 percent of the income, 

A single Lorenz curve can be evaluated in terms of its position relative to the 45° 
diagonal. The closer the curve is to the diagonal the greater the degree of equality, the more 
distant the curve is from the diagonal the more unequal the distribution. When two or more 
Lorenz curves are presented in the same diagram it is possible to compare them in terms 
of relative equality. The curves closer to the diagonal are always the more equal in their 
distribution of the characteristic bein^ examined, 

Lorenz curves have been adapted for a variety of uses in education. The units of 
observation can be individual students, classrooms, schools, geographical regions, etc, and 
the characteristics can be any of the input or output measures discussed here or th i outcome 
measures discussed in the succeeding session, Cohn (1979) indicates an adapla:ion of the 
Lorenz curve for school finance analysis. He presents the percent of students mnked by 
wealth relative to percent of total school expenditure on students. This is not a "true" 
Lorenz curve, however, since the characteristic measured (expenditures) is not the same as 
is used for the ranking of the students (district wealth). Another example of a modified 
Lorenz curve would be to rank schools by expenditure and relate this to cumulative 
■ achievement measures. As in the Cohn example (where one is testing the relationship of 
wealth to expenditure), one is comparing a determinant (school expenditure) to an effect 
(achievement).* 



♦ Heyneman and Loxley (1983A) indicate how Lorenz curves can be applied to educa- 
tional quality/educational achievement analysis among groups. 
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APPENDIX FIGURE TWO 

LORENZ CURVE OF EARNINGS 




It is always preferable to use a more traditionally determined Lorenz curve (where the 
characteristics used for measurement and ranking are the same) where possible because it 
increases the ease and clarity of interpretation. One may expect that the most common use 
of Lorenz curves in educational analysis will continue to be for comparing earnings differ- 
ences of different educational groups or cohorts. 

The final statistical measure of equality to be discussed is the Gini coefficient. The Gini 
coefficient was derived as a means of expressing numerically the relationships indicated by 
the Lo'^enz curve. Gini coefficients should only be used for "true" Lorenz curves (where 
the characteristic of incidence is the same as the criteria used for ranking the units of 
observation) and normally is not appropriate for adaptations of the type made by Cohn. 

The Gini coefficient represents the ratio of the area between the Lorenz curve and the 
diagonal to the area bounded by the diagonal and the lower horizontal and right vertical 
axis. In Appendix Figure Two, the coefficient would be equal to A divided by A + B. 
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S ince the value of A + B is constant for any given Lorcnz diagram . the change in the size 
of A (the total area separating the curve from the diagonal) will determine the size of the 
coefficient. The further the curve is away from the diagonal, the closer the value of A. 
approaches A + B. and the closer the coefficient value is to LO (absolute inequality). The 
closer the curve is to the diagonal, the lower the value of A. and the closer the coefficient 
value is to zero (absolute equality). Thus, a low coefficient value denotes greater equality 
and a high value denotes greater inequality. 

The Gini coefficient can allow for comparison of a larger number of disu ibutions at once 
and is not limited by the problem of space that one faces trying to draw multiple Lorenz 
curves in a fixed diagram size. As a result. Lorcnz curves now arc used more for 
pedagogical than analytical purposes in the study of inequality. 
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BIBLIOGRAPHY 



This selective, albeit extensive, bibliography >is indicative of the range of materials 
reviewed in the preparation of this document. Because the primary audience for the 
monograph is not the academic community, citations in the text have been kept to a 
minimum. However, the preparation of this bibliographic listing is intended to indicate the 
extent of the debt to, and the degree of influence exerted by, the educational and economic 
research communities. 

It should be noted that the lEES Project is listed as "author" of a series of reports and 
monographs. The reader should understand that the project is inclusive not just of its four 
member institutions (Florida State University, Howard University, the Institute for Inter- 
national Research, and the State University of New York at Albany) but incorporates the 
full membership of its professional counterparts in the member nations (Botswana, Haiti, 
Indonesia, Liberia, Nepal, Somalia, and the Yen: ?n Arab Republic). The majority of lEES 
documents are recognized as official government reports in the nations in which they were 
produced. 
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